Re: [VOTE] Release of Apache Phoenix 5.1.2 RC0

[la...@apache.org]

Hmm... Yeah, that's a problem if we do have enough PMCs to make a point release.

If one of the other PMCs happens to read this, please have a look and vote. :)





On Saturday, June 5, 2021, 8:04:04 AM PDT, Viraj Jasani  
wrote: 





Yes I again did basic local testing with RC and I have +1 vote, but mine
would be non-binding one.
Awaiting one more PMC binding vote.


On Fri, 4 Jun 2021 at 11:59 PM, la...@apache.org  wrote:

> So are we good?
>
> Assuming Viraj's +1 is implied we have three +1 and not -1.
>
> -- Lars
>
>
> On Wednesday, June 2, 2021, 7:10:58 AM PDT, Istvan Toth 
> wrote:
>
>
>
>
>
> +1 (binding)
>
> Checksum for source distribution: OK
> Signature for source distribution: OK
> Release notes and changes files look good: OK
> mvn clean apache-rat:check: OK
> mvn clean package: OK
> started phoenix_sandbox and ran smoke test: OK
>
> István
>
> On Mon, May 31, 2021 at 9:00 AM Viraj Jasani  wrote:
>
> > Please vote on this Apache phoenix release candidate, phoenix-5.1.2RC0
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache phoenix 5.1.2
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 5.1.2RC0:
> >
> >  https://github.com/apache/phoenix/tree/5.1.2RC0
> >
> > Commit ID: 5fe4931914a8d95f033644a9b48bc55e79a770f6
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.2RC0/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachephoenix-1238/
> >
> > Artifacts were signed with the 1C8ADFD5 key which can be found in:
> >
> >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> >
> > To learn more about Apache phoenix, please see
> >
> >  https://phoenix.apache.org/
> >
> > Thanks,
> > Your Phoenix Release Manager
> >
>

Re: [VOTE] Release of Apache Phoenix 5.1.2 RC0

[la...@apache.org]

So are we good?

Assuming Viraj's +1 is implied we have three +1 and not -1.

-- Lars


On Wednesday, June 2, 2021, 7:10:58 AM PDT, Istvan Toth  
wrote: 





+1 (binding)

Checksum for source distribution: OK
Signature for source distribution: OK
Release notes and changes files look good: OK
mvn clean apache-rat:check: OK
mvn clean package: OK
started phoenix_sandbox and ran smoke test: OK

István

On Mon, May 31, 2021 at 9:00 AM Viraj Jasani  wrote:

> Please vote on this Apache phoenix release candidate, phoenix-5.1.2RC0
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache phoenix 5.1.2
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 5.1.2RC0:
>
>  https://github.com/apache/phoenix/tree/5.1.2RC0
>
> Commit ID: 5fe4931914a8d95f033644a9b48bc55e79a770f6
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.2RC0/
>
> Maven artifacts are available in a staging repository at:
>
>
> https://repository.apache.org/content/repositories/orgapachephoenix-1238/
>
> Artifacts were signed with the 1C8ADFD5 key which can be found in:
>
>  https://dist.apache.org/repos/dist/release/phoenix/KEYS
>
> To learn more about Apache phoenix, please see
>
>  https://phoenix.apache.org/
>
> Thanks,
> Your Phoenix Release Manager
>

Re: [VOTE] Release of Apache Phoenix 5.1.2 RC0

[la...@apache.org]

+1 (binding)

Built from source, ran a local HDFS/ZK/HBase/Phoenix stack, inserted 26m rows, 
created some indexes, queried, tried the Trino-Phoenix connector.

All checks out.

-- Lars




On Monday, May 31, 2021, 12:01:07 AM PDT, Viraj Jasani  
wrote: 





Please vote on this Apache phoenix release candidate, phoenix-5.1.2RC0

The VOTE will remain open for at least 72 hours.

[ ] +1 Release this package as Apache phoenix 5.1.2
[ ] -1 Do not release this package because ...

The tag to be voted on is 5.1.2RC0:

  https://github.com/apache/phoenix/tree/5.1.2RC0

Commit ID: 5fe4931914a8d95f033644a9b48bc55e79a770f6

The release files, including signatures, digests, as well as CHANGES.md
and RELEASENOTES.md included in this RC can be found at:

  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.2RC0/

Maven artifacts are available in a staging repository at:

  https://repository.apache.org/content/repositories/orgapachephoenix-1238/

Artifacts were signed with the 1C8ADFD5 key which can be found in:

  https://dist.apache.org/repos/dist/release/phoenix/KEYS

To learn more about Apache phoenix, please see

  https://phoenix.apache.org/

Thanks,
Your Phoenix Release Manager

Re: Time for 5.1.2?

[la...@apache.org]

Nice!

I noticed we had some successful 5.1 runs too.
Fixing the tests would need some attention again. Flappers should either be 
fixed or disabled.

IMHO, a flapper is worse than a failing test because it undermines the 
confidence in a test run.
If the tests always pass you'll notice a failure right away.
But when most of the runs produce some kind of failure it is human nature to 
start ignoring them, and then new, real failures pile up.


-- Lars


On Friday, May 28, 2021, 11:36:53 PM PDT, Viraj Jasani  
wrote: 





Due to incorrect/missing fix versions, found some discrepancies b/ master
and 5.1 in the Pherf module.
Backported recent improvements to 5.1 including test fix:
PHOENIX-6417, PHOENIX-6118, PHOENIX-6430, PHOENIX-6429, PHOENIX-6431,
PHOENIX-6432

On Sat, May 29, 2021 at 12:20 AM Viraj Jasani  wrote:

> I looked at CI build today and there are a couple of flakies in core/pherf.
>
> 3 major flaky tests: PermissionNSEnabledIT, PermissionsCacheIT,
> AuditLoggingIT (reported with HBase 2.2/2.3/2.4). Permission tests have
> been flaky for quite some time. IIRC, there was some attempt to fix them as
> well but still they appear with AccessDeniedException at times. Not much
> sure about AuditLoggingIT though.
>
> Pherf test fails with HBase 2.1.
>
>
> On Fri, 28 May 2021 at 10:44 PM, la...@apache.org 
> wrote:
>
>> 5.1 CI builds seems to fail consistently - at least for the past two
>> weeks.
>>
>> https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/5.1/
>>
>> Might be something to look into.
>> I have an hour today between meetings, I'll see what I can do. :)
>>
>> -- Lars
>>
>>
>> On Wednesday, May 26, 2021, 10:54:13 PM PDT, Viraj Jasani <
>> vjas...@apache.org> wrote:
>>
>>
>>
>>
>>
>> Thanks Lars. I had an offline chat with Istvan yesterday and I will
>> prepare
>> RCs over the weekend.
>>
>> In the meantime, I will again take a look at Jira fixVersion and git
>> commits compatibility and also try to help with any backport (if still
>> pending).
>>
>>
>> On Wed, 26 May 2021 at 11:53 PM, la...@apache.org 
>> wrote:
>>
>> > I cherry-picked three changes into branch 5.1.
>> > Assuming Jira is up-to-date, there are no 4.16.x issues that are not
>> also
>> > in 5.1.x.
>> >
>> > There are 12 issues left that are in 4.17.0 and 5.2.0, but not in 5.1.x.
>> > But they are presumably not in 4.16.x because they violate backward
>> > compatibility, and - if so - should not be in 5.1.2.
>> >
>> > -- Lars
>> >
>> > On Wednesday, May 26, 2021, 10:15:31 AM PDT, la...@apache.org <
>> > la...@apache.org> wrote:
>> >
>> >
>> >
>> >
>> >
>> > Awesome. Thanks Istvan.
>> >
>> > I'll do a pass through the issues too.
>> >
>> > -- Lars
>> >
>> >
>> > On Tuesday, May 25, 2021, 8:45:00 PM PDT, Istvan Toth > >
>> > wrote:
>> >
>> >
>> >
>> >
>> >
>> > I am not aware of any blockers in 5.1.
>> > If there is no objection, and no other volunteers for the 5.1.2 RM
>> role, I
>> > intend to start the process on Monday.
>> >
>> > I will also look for missing fixes before release, but if anyone is
>> aware
>> > of any fix that is needed in 5.1.2 , but
>> > hasn't been backported, then please try to get it resolved by Monday.
>> >
>> > regards
>> > Istvan
>> >
>> > On Wed, May 26, 2021 at 4:19 AM la...@apache.org 
>> wrote:
>> >
>> > > There are 17 resolved issues against it.
>> > > PHOENIX-6436 is needed for further integration with Trino (formerly
>> > > Presto).
>> > >
>> > > It is time for another point release?
>> > >
>> > > -- Lars
>> > >
>> >
>>
>

Re: Time for 5.1.2?

[la...@apache.org]

5.1 CI builds seems to fail consistently - at least for the past two weeks.

https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/5.1/

Might be something to look into.
I have an hour today between meetings, I'll see what I can do. :)

-- Lars


On Wednesday, May 26, 2021, 10:54:13 PM PDT, Viraj Jasani  
wrote: 





Thanks Lars. I had an offline chat with Istvan yesterday and I will prepare
RCs over the weekend.

In the meantime, I will again take a look at Jira fixVersion and git
commits compatibility and also try to help with any backport (if still
pending).


On Wed, 26 May 2021 at 11:53 PM, la...@apache.org  wrote:

> I cherry-picked three changes into branch 5.1.
> Assuming Jira is up-to-date, there are no 4.16.x issues that are not also
> in 5.1.x.
>
> There are 12 issues left that are in 4.17.0 and 5.2.0, but not in 5.1.x.
> But they are presumably not in 4.16.x because they violate backward
> compatibility, and - if so - should not be in 5.1.2.
>
> -- Lars
>
> On Wednesday, May 26, 2021, 10:15:31 AM PDT, la...@apache.org <
> la...@apache.org> wrote:
>
>
>
>
>
> Awesome. Thanks Istvan.
>
> I'll do a pass through the issues too.
>
> -- Lars
>
>
> On Tuesday, May 25, 2021, 8:45:00 PM PDT, Istvan Toth 
> wrote:
>
>
>
>
>
> I am not aware of any blockers in 5.1.
> If there is no objection, and no other volunteers for the 5.1.2 RM role, I
> intend to start the process on Monday.
>
> I will also look for missing fixes before release, but if anyone is aware
> of any fix that is needed in 5.1.2 , but
> hasn't been backported, then please try to get it resolved by Monday.
>
> regards
> Istvan
>
> On Wed, May 26, 2021 at 4:19 AM la...@apache.org  wrote:
>
> > There are 17 resolved issues against it.
> > PHOENIX-6436 is needed for further integration with Trino (formerly
> > Presto).
> >
> > It is time for another point release?
> >
> > -- Lars
> >
>

Re: [DISCUSS] Unbundling Sqlline and slf4j backend from phoenix-client and phoenix-client embedded

[la...@apache.org]

- User

I see, make sense. So if I understand this correctly then:

1. Remove sqlline et al from phoenix-client
2. Remove phoenix-client embedded (since it is now the same as phoenix-client)
3. Provide an uber sqlline jar, for use with sqlline.py.

i.e. option #1?

PR 1239 seems to keep the phoenix-client jar, though. Which is option #2 below.

 -- Lars

On Wednesday, May 26, 2021, 9:36:30 PM PDT, Istvan Toth  
wrote: 





The technical details:

The plan is to ship the sqlline ubjerjar (sqlline provides an uberjar with all 
of its own dependencies shaded in)
in the /lib directory, along with the log4j slf4j  backend jar.
sqlline.py and friends is to be updated to add those to the classpath.

Richard has this already up in https://github.com/apache/phoenix/pull/1239

On Wed, May 26, 2021 at 9:43 PM Josh Elser  wrote:
> I think the idea is that we would include a sqlline jar with the Phoenix 
> distribution. Context: we had some grief where a sqlline upgrade caused 
> user pain because they were relying on specific output from sqlline.
> 
> If we have the sqlline jar _not_ packaged inside phoenix-client, then 
> users can easily replace the version of sqlline which makes them happiest.
> 
> While I agree with Istvan that #1 is the more "correct" option, I'm 
> worried about the impact of folks who rely on the phoenix-client.jar to 
> be a "batteries included" fat-jar. Removing sqlline from 
> phoenix-client-embedded is great, so I'd lean towards #2.
> 
> We can see what adoption of phoenix-client-embedded looks like now that 
> we have it in releases. I imagine most folks haven't yet realized that 
> it's even an option that's available.
> 
> On 5/26/21 1:16 PM, la...@apache.org wrote:
>> Will sqlline still be part of the Phoenix "distribution"? Or will it become 
>> a separate package to install?
>> 
>> 
>> 
>> 
>> 
>> 
>> On Wednesday, May 26, 2021, 1:07:17 AM PDT, Istvan Toth  
>> wrote:
>> 
>> 
>> 
>> 
>> 
>> Hi!
>> 
>> The current purpose of the phoenix-client JAR is twofold:
>> - It servers as a generic JDBC driver for embedding in applications
>> - It also contains the sqlline library used by the sqlline.py script, as
>> well as the slf4j log4j backend.
>> - (It also contains a some Phoenix code and HBase libraries not necessary
>> for a client, but we're already tracking that in different tickets)
>> 
>> One major pain point is the slf4j backend, which makes phoenix-client
>> incompatible with applications and libraries that do not use log4j 1.2 as a
>> backend, and kind of defeats the purpose of using slf4j in the first place.
>> phoenix-client-embedded solves this problem by removing the slf4j backend
>> from Phoenix.
>> 
>> In PHOENIX-6378 <https://issues.apache.org/jira/browse/PHOENIX-6378> we aim
>> to remove sqlline from the phoenix-client JAR, as it further cleans up the
>> classpath, and avoids locking phoenix to the sqlline version that it was
>> built with.
>> 
>> In Richard's current patch, we remove sqlline from phoenix-client-embedded,
>> and use that in the sqlline script.
>> 
>> In our quest for a more useable phoenix-client, we can do two things now:
>> 
>>    1. Remove both the slf4j backend, and sqlline from phoenix-client, and
>>    also drop phoenix-client-embedded as it would be the same as 
>>phoenix-client
>>    2. Remove sqlline from phoenix-client-embedded, and keep the current
>>    phoenix-client as backwards compatibility option
>> 
>> I'd prefer the first option, but this is somewhat more disruptive than the
>> other.
>> 
>> Please share your thoughts. Do you prefer option 1, 2, or something else
>> entirely ?
>> 
>> Istvan
>> 
> 


-- 
István Tóth  | Staff Software Engineer 

st...@cloudera.com  


 

Re: Time for 5.1.2?

[la...@apache.org]

I cherry-picked three changes into branch 5.1.
Assuming Jira is up-to-date, there are no 4.16.x issues that are not also in 
5.1.x.

There are 12 issues left that are in 4.17.0 and 5.2.0, but not in 5.1.x. But 
they are presumably not in 4.16.x because they violate backward compatibility, 
and - if so - should not be in 5.1.2. 

-- Lars

On Wednesday, May 26, 2021, 10:15:31 AM PDT, la...@apache.org 
 wrote: 





Awesome. Thanks Istvan.

I'll do a pass through the issues too.

-- Lars


On Tuesday, May 25, 2021, 8:45:00 PM PDT, Istvan Toth  wrote: 





I am not aware of any blockers in 5.1.
If there is no objection, and no other volunteers for the 5.1.2 RM role, I
intend to start the process on Monday.

I will also look for missing fixes before release, but if anyone is aware
of any fix that is needed in 5.1.2 , but
hasn't been backported, then please try to get it resolved by Monday.

regards
Istvan

On Wed, May 26, 2021 at 4:19 AM la...@apache.org  wrote:

> There are 17 resolved issues against it.
> PHOENIX-6436 is needed for further integration with Trino (formerly
> Presto).
>
> It is time for another point release?
>
> -- Lars
>

Re: [DISCUSS] Unbundling Sqlline and slf4j backend from phoenix-client and phoenix-client embedded

[la...@apache.org]

Will sqlline still be part of the Phoenix "distribution"? Or will it become a 
separate package to install?






On Wednesday, May 26, 2021, 1:07:17 AM PDT, Istvan Toth  
wrote: 





Hi!

The current purpose of the phoenix-client JAR is twofold:
- It servers as a generic JDBC driver for embedding in applications
- It also contains the sqlline library used by the sqlline.py script, as
well as the slf4j log4j backend.
- (It also contains a some Phoenix code and HBase libraries not necessary
for a client, but we're already tracking that in different tickets)

One major pain point is the slf4j backend, which makes phoenix-client
incompatible with applications and libraries that do not use log4j 1.2 as a
backend, and kind of defeats the purpose of using slf4j in the first place.
phoenix-client-embedded solves this problem by removing the slf4j backend
from Phoenix.

In PHOENIX-6378  we aim
to remove sqlline from the phoenix-client JAR, as it further cleans up the
classpath, and avoids locking phoenix to the sqlline version that it was
built with.

In Richard's current patch, we remove sqlline from phoenix-client-embedded,
and use that in the sqlline script.

In our quest for a more useable phoenix-client, we can do two things now:

  1. Remove both the slf4j backend, and sqlline from phoenix-client, and
  also drop phoenix-client-embedded as it would be the same as phoenix-client
  2. Remove sqlline from phoenix-client-embedded, and keep the current
  phoenix-client as backwards compatibility option

I'd prefer the first option, but this is somewhat more disruptive than the
other.

Please share your thoughts. Do you prefer option 1, 2, or something else
entirely ?

Istvan

Re: Time for 5.1.2?

[la...@apache.org]

Awesome. Thanks Istvan.

I'll do a pass through the issues too.

-- Lars


On Tuesday, May 25, 2021, 8:45:00 PM PDT, Istvan Toth  wrote: 





I am not aware of any blockers in 5.1.
If there is no objection, and no other volunteers for the 5.1.2 RM role, I
intend to start the process on Monday.

I will also look for missing fixes before release, but if anyone is aware
of any fix that is needed in 5.1.2 , but
hasn't been backported, then please try to get it resolved by Monday.

regards
Istvan

On Wed, May 26, 2021 at 4:19 AM la...@apache.org  wrote:

> There are 17 resolved issues against it.
> PHOENIX-6436 is needed for further integration with Trino (formerly
> Presto).
>
> It is time for another point release?
>
> -- Lars
>

Time for 5.1.2?

[la...@apache.org]

There are 17 resolved issues against it.
PHOENIX-6436 is needed for further integration with Trino (formerly Presto).

It is time for another point release?

-- Lars

Re: Don't forget about branch-5.1

[la...@apache.org]

add columns to
> System.Catalog and hence can't go out in 4.16.2 or 5.1.2. (See PHOENIX-6247
> -- which should be marked Resolved -- and PHOENIX-6457).
>
> Do we need the dual maintenance of a 5.1 patch branch?
>
> Geoffrey
>
> On Wed, May 12, 2021 at 12:55 PM la...@apache.org 
> wrote:
>
> > Hi all,
> >
> > I noticed some changes that went into both master and the 4.x branch, but
> > not into branch-5.1.
> >
> > Unfortunately I do not have time to check each edit.
> >
> > When you commit changes, please do not forget about branch-5.1, which
> will
> > produce our next version of Phoenix.
> >
> > Cheers.
> >
> > -- Lars

> >
>


-- 
*István Tóth* | Staff Software Engineer
st...@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/
>
--

Don't forget about branch-5.1

[la...@apache.org]

Hi all,

I noticed some changes that went into both master and the 4.x branch, but not 
into branch-5.1.

Unfortunately I do not have time to check each edit.

When you commit changes, please do not forget about branch-5.1, which will 
produce our next version of Phoenix.

Cheers.

-- Lars

Re: [DISCUSS] Separating client and server side code

[la...@apache.org]

There is also another angle to look at. A long time ago I wrote this:

"
It seems Phoenix serves 4 distinct purposes:
1. Query parsing and compiling.
2. A type system
3. Query execution
4. Efficient HBase interface

Each of these is useful by itself, but we do not expose these as stable 
interfaces.
We have seen a lot of need to tie HBase into "higher level" service, such as 
Spark (and Presto, etc).
I think we can get a long way if we separate at least #1 (SQL) from the rest 
#2, #3, and #4 (Typed HBase Interface - THI).
Phoenix is used via SQL (#1), other tools such as Presto, Impala, Drill, Spark, 
etc, can interface efficiently with HBase via THI (#2, #3, and #4).
"

I still believe this is an additional useful demarcation for how to group the 
code. And coincided somewhat with server/client.

Query parsing and the type system are client. Query execution and HBase 
interface are both client and server.

-- Lars

On Wednesday, April 14, 2021, 8:56:08 AM PDT, Istvan Toth  
wrote: 





Jacob, Josh and me had a discussion about the topic.

I'm attaching the dependency graph of the proposed modules



On Fri, Apr 9, 2021 at 6:30 AM Istvan Toth  wrote:
> The bulk of the changes I'm working on is indeed the separation of the client 
> and the server side code.
> 
> Separating the MR related classes, and the tools-specific code (main, options 
> parsing, etc) makes sense to me, if we don't mind adding another module.
> 
> In the first WIP iteration, I'm splitting out everything that depends on more 
> than hbase-client into a "server" module.
> Once that works I will look at splitting that further into a  real "server" 
> and an "MR/tools" module.
> 
> 
> My initial estimates about splitting the server side code were way too 
> optimistic, we have to touch a lot of code to break circular dependencies 
> between the client and server side. The changes are still quite trivial, but 
> the patch is going to be huge and scary.
> 
> 
> Tests are also going to be a problem, we're probably going to have to move 
> most of them into the "server" or a separate "tests" module, as the 
> MiniCluster tests depend on code from each module.
> 
> The plan in PHOENIX-5483, and Lars's mail sounds good, but I think that it 
> would be more about dividing the "client-side" module further. 
> (BTW I think that making the indexing engine available separately would also 
> be a popular feature )
> 
> 
> 
> On Fri, Apr 9, 2021 at 5:39 AM Daniel Wong  wrote:
>> This is another project I am interested in as well as my group at
>> Salesforce.  We have had some discussions internally on this but I wasn't
>> aware of this specific Spark issue (We only allow phoenix access via spark
>> by default).  I think the approaches outlined are a good initial step but
>> we were also considering a larger breakup of phoenix-core.  I don't
>> think the desire for the larger step should stop us from doing the initial
>> ones Istavan and Josh proposed.  I think the high level plan makes sense
>> but I might prefer a different name than phoenix-tools for the ones we want
>> to be available to external libraries like phoenix-connectors.  Another
>> possible alternative is to restructure maybe less invasively by making
>> phoenix core like your proposed tools and making a phoenix-internal or
>> similar for the future.
>> One thing I was wondering was how much effort it was to split client/server
>> through phoenix-core...  Lars layed out a good component view of phoenix
>> whosethe first step might be PHOENIx-5483 but we could focus on highest
>> level separation rather than bottom up.  However, even that thread linked
>> there talks about a client-facing api which we can piggyback for this use.
>> Say phoeinx-public-api or similar.
>> 
>> On Wed, Apr 7, 2021 at 9:43 AM Jacob Isaac  wrote:
>> 
>>> Hi Josh & Istvan
>>>
>>> Thanks Istvan for looking into this, I am also interested in solving this
>>> problem,
>>> Let me know how I can help?
>>>
>>> Thanks
>>> Jacob
>>>
>>> On Wed, Apr 7, 2021 at 9:05 AM Josh Elser  wrote:
>>>
>>> > Thanks for trying to tackle this sticky problem, Istvan. For the context
>>> > of everyone else, the real-life problem Istvan is trying to fix is that
>>> > you cannot run a Spark application with both HBase and Phoenix jars on
>>> > the classpath.
>>> >
>>> > If I understand this correctly, it's that the HBase API signatures are
>>> > different depending on whether we are "client side" or "server side"
>>> > (within a RegionServer). Your comment on PHOENIX-6053 shows that
>>> > (signatures on Table.java around Protobuf's Service class having shaded
>>> > relocation vs. the original com.google.protobuf coordinates).
>>> >
>>> > I think the reason we have the monolithic phoenix-core is that we have
>>> > so much logic which is executed on both the client and server side. For
>>> > example, we may push a filter operation to the server-side or we many
>>> > run it client-side. That's also why we have the "thin" phoenix-server
>>> > 

Re: On duplicate column names

[la...@apache.org]

Please comment on the Jira if you have an opinion :)






On Monday, March 29, 2021, 10:48:34 AM PDT, la...@apache.org  
wrote: 





Filed https://issues.apache.org/jira/browse/PHOENIX-6433

We can discuss there.
In the end, IMHO, placing columns in distinct column families is a physical 
optimization, not a logical construct...
Very much like indexes. And just like indexes these physical optimizations 
should not leak into the actual queries.

-- Lars


On Monday, March 29, 2021, 3:04:08 AM PDT, Istvan Toth  
wrote: 





I agree that having identical column names is generally a pain, and I have
no problem with disallowing them in statically defined Phoenix columns, as
long as the views on HBase tables and the dynamic column features are not
affected.

We may or may not want to add a property to override it (with the
default being disallowing the duplicate column names)

Istvan



On Sat, Mar 27, 2021 at 5:59 PM la...@apache.org  wrote:

> Viraj, yeah similar discussion.
>
> Istvan, good point. Of course you cannot guard against what folks do at
> the HBase level, and we should still allow that. Just like in PHOENIX-6343.
> But we could disallow creating new tables like this in Phoenix, also like
> PHOENIX-6343.
>
>
> I just think that causes more problems than it solves. While HBase looks
> at things key-value by key-value, Phoenix takes a row-by-row view.
> From that angle, why would you ever want a table with ambiguous column
> names? And neither qualified nor duplicate column names are in the SQL
> standard - AFAIK at least.
>
> In the Trino case I described, it's using the standard JDBC metadata, then
> using the COLUMN_NAM column to read the name of the column. That is the
> JDBC standard.
> In the Phoenix case you now also have to read the COLUMN_FAMILY column,
> then know how to build a qualified column name, and that be able to pass
> this through all the query planning, etc. And there is no possible way to
> hand a qualified column name to Phoenix. Phoenix does *not* allow "cf.cq",
> it has to be cf.cq or "cf"."cq".
> Apparently there's a way in Trino to model this as a structured column.
> But that too does not hit the mark, then you *have* to use structs to
> access any column in Phoenix.
>
> So in Trino (and probably other JDBC clients) we could opt to simply fail
> on tables like these - that's what's happening now... Or come up with
> unnatural constructs to fit this into the SQL model. Trino is just an
> example here.
>
> At least we can document that qualifier names *should* be unique, and it
> otherwise might cause problems with downstream BigData integrations.
>
> -- Lars
>
> On Saturday, March 27, 2021, 2:51:04 AM PDT, Viraj Jasani <
> vjas...@apache.org> wrote:
>
>
>
>
>
> Somewhat similar discussion we had on PHOENIX-6343 and why the duplicate
> column check was restricted to default CF only.
>
>
> On Sat, 27 Mar 2021 at 10:42 AM, Istvan Toth 
> wrote:
>
> > The first thing that comes to my mind is that this would limit
> > functionality when defining views on existing raw HBase tables (though
> > aliasing the columns may solve that)
> > Another thing to consider is how this would affect dynamic column use
> cases
> > for either native Phoenix tables of views on HBase tables.
> >
> > Istvan
> >
> > On Sat, Mar 27, 2021 at 12:20 AM la...@apache.org 
> > wrote:
> >
> > > As you may or may not know, Phoenix allows for duplicate column names
> as
> > > long as they are placed in different column families.
> > >
> > > You can create a table such as CREATE TABLE t (pk1 ..., x.v1, y.v1,
> ...).
> > > Now each time you want to refer to v1 you need to qualify it with its
> > > column family or you get a AmbiguousColumnException.
> > >
> > > Worse, you also do CREATE TABLE t (pk1 ..., v1, x.v1, ...). Now a using
> > v1
> > > in a query will silently resolve to v1 in the default column family and
> > > x.v1 does have to be qualified.
> > >
> > > As I reason through how this should work in Trino's (formerly Presto)
> > > Phoenix connector, it occurs to me that should probably just disallow
> > this.
> > >
> > > CREATE TABLE t (pk1 ..., x.v1, y.v2, ...) works just fine and you can
> > > refer to both v1 and v2 without the need to qualify them with the
> column
> > > family.
> > >
> > > So... In essence: Should we require all columns to be uniquely named,
> > even
> > > with multiple column families?
> > >
> > > Thanks.
> > >
> > > -- Lars
> > >
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > st...@cloudera.com <https://www.cloudera.com>
> > [image: Cloudera] <https://www.cloudera.com/>
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > <https://www.cloudera.com/>
> > --
> >
>

Re: On duplicate column names

[la...@apache.org]

Filed https://issues.apache.org/jira/browse/PHOENIX-6433

We can discuss there.
In the end, IMHO, placing columns in distinct column families is a physical 
optimization, not a logical construct...
Very much like indexes. And just like indexes these physical optimizations 
should not leak into the actual queries.

-- Lars


On Monday, March 29, 2021, 3:04:08 AM PDT, Istvan Toth  
wrote: 





I agree that having identical column names is generally a pain, and I have
no problem with disallowing them in statically defined Phoenix columns, as
long as the views on HBase tables and the dynamic column features are not
affected.

We may or may not want to add a property to override it (with the
default being disallowing the duplicate column names)

Istvan



On Sat, Mar 27, 2021 at 5:59 PM la...@apache.org  wrote:

> Viraj, yeah similar discussion.
>
> Istvan, good point. Of course you cannot guard against what folks do at
> the HBase level, and we should still allow that. Just like in PHOENIX-6343.
> But we could disallow creating new tables like this in Phoenix, also like
> PHOENIX-6343.
>
>
> I just think that causes more problems than it solves. While HBase looks
> at things key-value by key-value, Phoenix takes a row-by-row view.
> From that angle, why would you ever want a table with ambiguous column
> names? And neither qualified nor duplicate column names are in the SQL
> standard - AFAIK at least.
>
> In the Trino case I described, it's using the standard JDBC metadata, then
> using the COLUMN_NAM column to read the name of the column. That is the
> JDBC standard.
> In the Phoenix case you now also have to read the COLUMN_FAMILY column,
> then know how to build a qualified column name, and that be able to pass
> this through all the query planning, etc. And there is no possible way to
> hand a qualified column name to Phoenix. Phoenix does *not* allow "cf.cq",
> it has to be cf.cq or "cf"."cq".
> Apparently there's a way in Trino to model this as a structured column.
> But that too does not hit the mark, then you *have* to use structs to
> access any column in Phoenix.
>
> So in Trino (and probably other JDBC clients) we could opt to simply fail
> on tables like these - that's what's happening now... Or come up with
> unnatural constructs to fit this into the SQL model. Trino is just an
> example here.
>
> At least we can document that qualifier names *should* be unique, and it
> otherwise might cause problems with downstream BigData integrations.
>
> -- Lars
>
> On Saturday, March 27, 2021, 2:51:04 AM PDT, Viraj Jasani <
> vjas...@apache.org> wrote:
>
>
>
>
>
> Somewhat similar discussion we had on PHOENIX-6343 and why the duplicate
> column check was restricted to default CF only.
>
>
> On Sat, 27 Mar 2021 at 10:42 AM, Istvan Toth 
> wrote:
>
> > The first thing that comes to my mind is that this would limit
> > functionality when defining views on existing raw HBase tables (though
> > aliasing the columns may solve that)
> > Another thing to consider is how this would affect dynamic column use
> cases
> > for either native Phoenix tables of views on HBase tables.
> >
> > Istvan
> >
> > On Sat, Mar 27, 2021 at 12:20 AM la...@apache.org 
> > wrote:
> >
> > > As you may or may not know, Phoenix allows for duplicate column names
> as
> > > long as they are placed in different column families.
> > >
> > > You can create a table such as CREATE TABLE t (pk1 ..., x.v1, y.v1,
> ...).
> > > Now each time you want to refer to v1 you need to qualify it with its
> > > column family or you get a AmbiguousColumnException.
> > >
> > > Worse, you also do CREATE TABLE t (pk1 ..., v1, x.v1, ...). Now a using
> > v1
> > > in a query will silently resolve to v1 in the default column family and
> > > x.v1 does have to be qualified.
> > >
> > > As I reason through how this should work in Trino's (formerly Presto)
> > > Phoenix connector, it occurs to me that should probably just disallow
> > this.
> > >
> > > CREATE TABLE t (pk1 ..., x.v1, y.v2, ...) works just fine and you can
> > > refer to both v1 and v2 without the need to qualify them with the
> column
> > > family.
> > >
> > > So... In essence: Should we require all columns to be uniquely named,
> > even
> > > with multiple column families?
> > >
> > > Thanks.
> > >
> > > -- Lars
> > >
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > st...@cloudera.com <https://www.cloudera.com>
> > [image: Cloudera] <https://www.cloudera.com/>
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > <https://www.cloudera.com/>
> > --
> >
>

Re: On duplicate column names

[la...@apache.org]

Viraj, yeah similar discussion.

Istvan, good point. Of course you cannot guard against what folks do at the 
HBase level, and we should still allow that. Just like in PHOENIX-6343.
But we could disallow creating new tables like this in Phoenix, also like 
PHOENIX-6343.


I just think that causes more problems than it solves. While HBase looks at 
things key-value by key-value, Phoenix takes a row-by-row view.
>From that angle, why would you ever want a table with ambiguous column names? 
>And neither qualified nor duplicate column names are in the SQL standard - 
>AFAIK at least.

In the Trino case I described, it's using the standard JDBC metadata, then 
using the COLUMN_NAM column to read the name of the column. That is the JDBC 
standard.
In the Phoenix case you now also have to read the COLUMN_FAMILY column, then 
know how to build a qualified column name, and that be able to pass this 
through all the query planning, etc. And there is no possible way to hand a 
qualified column name to Phoenix. Phoenix does *not* allow "cf.cq", it has to 
be cf.cq or "cf"."cq".
Apparently there's a way in Trino to model this as a structured column. But 
that too does not hit the mark, then you *have* to use structs to access any 
column in Phoenix.

So in Trino (and probably other JDBC clients) we could opt to simply fail on 
tables like these - that's what's happening now... Or come up with unnatural 
constructs to fit this into the SQL model. Trino is just an example here.

At least we can document that qualifier names *should* be unique, and it 
otherwise might cause problems with downstream BigData integrations.

-- Lars

On Saturday, March 27, 2021, 2:51:04 AM PDT, Viraj Jasani  
wrote: 





Somewhat similar discussion we had on PHOENIX-6343 and why the duplicate
column check was restricted to default CF only.


On Sat, 27 Mar 2021 at 10:42 AM, Istvan Toth 
wrote:

> The first thing that comes to my mind is that this would limit
> functionality when defining views on existing raw HBase tables (though
> aliasing the columns may solve that)
> Another thing to consider is how this would affect dynamic column use cases
> for either native Phoenix tables of views on HBase tables.
>
> Istvan
>
> On Sat, Mar 27, 2021 at 12:20 AM la...@apache.org 
> wrote:
>
> > As you may or may not know, Phoenix allows for duplicate column names as
> > long as they are placed in different column families.
> >
> > You can create a table such as CREATE TABLE t (pk1 ..., x.v1, y.v1, ...).
> > Now each time you want to refer to v1 you need to qualify it with its
> > column family or you get a AmbiguousColumnException.
> >
> > Worse, you also do CREATE TABLE t (pk1 ..., v1, x.v1, ...). Now a using
> v1
> > in a query will silently resolve to v1 in the default column family and
> > x.v1 does have to be qualified.
> >
> > As I reason through how this should work in Trino's (formerly Presto)
> > Phoenix connector, it occurs to me that should probably just disallow
> this.
> >
> > CREATE TABLE t (pk1 ..., x.v1, y.v2, ...) works just fine and you can
> > refer to both v1 and v2 without the need to qualify them with the column
> > family.
> >
> > So... In essence: Should we require all columns to be uniquely named,
> even
> > with multiple column families?
> >
> > Thanks.
> >
> > -- Lars
> >
>
>
> --
> *István Tóth* | Staff Software Engineer
> st...@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> --
>

On duplicate column names

[la...@apache.org]

As you may or may not know, Phoenix allows for duplicate column names as long 
as they are placed in different column families.

You can create a table such as CREATE TABLE t (pk1 ..., x.v1, y.v1, ...).
Now each time you want to refer to v1 you need to qualify it with its column 
family or you get a AmbiguousColumnException.

Worse, you also do CREATE TABLE t (pk1 ..., v1, x.v1, ...). Now a using v1 in a 
query will silently resolve to v1 in the default column family and x.v1 does 
have to be qualified.

As I reason through how this should work in Trino's (formerly Presto) Phoenix 
connector, it occurs to me that should probably just disallow this.

CREATE TABLE t (pk1 ..., x.v1, y.v2, ...) works just fine and you can refer to 
both v1 and v2 without the need to qualify them with the column family.

So... In essence: Should we require all columns to be uniquely named, even with 
multiple column families?

Thanks.

-- Lars

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC2

[la...@apache.org]

I maintain my +1 for 5.1.1RC2.




On Sunday, March 21, 2021, 10:11:38 PM PDT, Istvan Toth  
wrote: 





Hi!

I'm slightly leaning towards getting the more impactful fixes already in
5.1.1 RC2 released, rather than iterating more on it, but for now I'm going
to take a passive role, and leave the decision to the PMC.
We have two binding +2s at the moment, so I'll just follow procedure, and
make the decision based on the next binding vote.

Istvan

On Sat, Mar 20, 2021 at 7:30 PM la...@apache.org  wrote:

> PHOENIX-6423 could be blocker...?
>
> What happens is that SELECT * queries with mixed default and explicit
> column families would return the wrong set of columns.
> And in some situation the WHERE clause evaluation would fail.
>
> I'm fine either way.
>
> -- Lars
>
>
>
> On Friday, March 19, 2021, 7:04:08 PM PDT, la...@apache.org <
> la...@apache.org> wrote:
>
>
>
>
>
> If 5.1.1 is sunk for any reason, I'd like to get PHOENIX-6421 in. (By
> itself it's not enough to sink the release, I think)
>
>
> On Wednesday, March 17, 2021, 8:53:59 PM PDT, Xinyi Yan <
> yanxi...@apache.org> wrote:
>
>
>
>
>
> +1 (non-binding)
>
> Besides the signature, checksum, and mvn install/verify, I built from the
> source and loaded the jar to my local environment. Create table/view/tenant
> view, upsert and select queries are working as expected.
>
> On Wed, Mar 17, 2021 at 8:32 PM Viraj Jasani  wrote:
>
> > +1 (non-binding)
> >
> > * Signature: ok
> > * Checksum : ok
> > * Rat check (1.8.0_171): ok
> >  - mvn clean apache-rat:check
> > * Built from source (1.8.0_171): ok
> >  - mvn clean install  -DskipTests
> > * Basic CRUD testing locally: ok
> > * Unit tests pass (1.8.0_171): failed (few flakes, passed in subsequent
> > run)
> >  - mvn clean package  && mvn verify  -Dskip.embedded
> >
> >
> > On Tue, 16 Mar 2021 at 8:58 PM, Istvan Toth  wrote:
> >
> > > Please vote on this Apache phoenix release candidate,
> > > phoenix-5.1.1RC2
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache phoenix 5.1.1
> > > [ ] -1 Do not release this package because ...
> > >
> > > The tag to be voted on is 5.1.1RC2:
> > >
> > >  https://github.com/apache/phoenix/tree/5.1.1RC2
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC2/
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >  https://repository.apache.org/#stagingRepositories
> > > (orgapachephoenix-1229)
> > >
> > > Artifacts were signed with the 0x794433C7 key which can be found in:
> > >
> > >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> > >
> > > To learn more about Apache phoenix, please see
> > >
> > >  http://phoenix.apache.org/
> > >
> > > Thanks,
> > > Istvan
> > >
> >
>

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC2

[la...@apache.org]

PHOENIX-6423 could be blocker...?

What happens is that SELECT * queries with mixed default and explicit column 
families would return the wrong set of columns.
And in some situation the WHERE clause evaluation would fail.

I'm fine either way.

-- Lars



On Friday, March 19, 2021, 7:04:08 PM PDT, la...@apache.org  
wrote: 





If 5.1.1 is sunk for any reason, I'd like to get PHOENIX-6421 in. (By itself 
it's not enough to sink the release, I think)


On Wednesday, March 17, 2021, 8:53:59 PM PDT, Xinyi Yan  
wrote: 





+1 (non-binding)

Besides the signature, checksum, and mvn install/verify, I built from the
source and loaded the jar to my local environment. Create table/view/tenant
view, upsert and select queries are working as expected.

On Wed, Mar 17, 2021 at 8:32 PM Viraj Jasani  wrote:

> +1 (non-binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_171): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_171): ok
>  - mvn clean install  -DskipTests
> * Basic CRUD testing locally: ok
> * Unit tests pass (1.8.0_171): failed (few flakes, passed in subsequent
> run)
>  - mvn clean package  && mvn verify  -Dskip.embedded
>
>
> On Tue, 16 Mar 2021 at 8:58 PM, Istvan Toth  wrote:
>
> > Please vote on this Apache phoenix release candidate,
> > phoenix-5.1.1RC2
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache phoenix 5.1.1
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 5.1.1RC2:
> >
> >  https://github.com/apache/phoenix/tree/5.1.1RC2
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC2/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >  https://repository.apache.org/#stagingRepositories
> > (orgapachephoenix-1229)
> >
> > Artifacts were signed with the 0x794433C7 key which can be found in:
> >
> >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> >
> > To learn more about Apache phoenix, please see
> >
> >  http://phoenix.apache.org/
> >
> > Thanks,
> > Istvan
> >
>

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC2

[la...@apache.org]

If 5.1.1 is sunk for any reason, I'd like to get PHOENIX-6421 in. (By itself 
it's not enough to sink the release, I think)


On Wednesday, March 17, 2021, 8:53:59 PM PDT, Xinyi Yan  
wrote: 





+1 (non-binding)

Besides the signature, checksum, and mvn install/verify, I built from the
source and loaded the jar to my local environment. Create table/view/tenant
view, upsert and select queries are working as expected.

On Wed, Mar 17, 2021 at 8:32 PM Viraj Jasani  wrote:

> +1 (non-binding)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_171): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_171): ok
>  - mvn clean install  -DskipTests
> * Basic CRUD testing locally: ok
> * Unit tests pass (1.8.0_171): failed (few flakes, passed in subsequent
> run)
>  - mvn clean package  && mvn verify  -Dskip.embedded
>
>
> On Tue, 16 Mar 2021 at 8:58 PM, Istvan Toth  wrote:
>
> > Please vote on this Apache phoenix release candidate,
> > phoenix-5.1.1RC2
> >
> > The VOTE will remain open for at least 72 hours.
> >
> > [ ] +1 Release this package as Apache phoenix 5.1.1
> > [ ] -1 Do not release this package because ...
> >
> > The tag to be voted on is 5.1.1RC2:
> >
> >  https://github.com/apache/phoenix/tree/5.1.1RC2
> >
> > The release files, including signatures, digests, as well as CHANGES.md
> > and RELEASENOTES.md included in this RC can be found at:
> >
> >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC2/
> >
> > Maven artifacts are available in a staging repository at:
> >
> >  https://repository.apache.org/#stagingRepositories
> > (orgapachephoenix-1229)
> >
> > Artifacts were signed with the 0x794433C7 key which can be found in:
> >
> >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> >
> > To learn more about Apache phoenix, please see
> >
> >  http://phoenix.apache.org/
> >
> > Thanks,
> > Istvan
> >
>

Re: Topic suggestions for April Tech Talk

[la...@apache.org]

An interesting topic would BigData ecosystem integration, which is clearly one 
of the strengths of Phoenix.

I.e. with Trino (formerly Presto), Spark, and classical MapReduce. Perhaps this 
could cover also HBase snapshot support.

I'm happy to cover this. But I can't to that for April 1st - juggling too many 
things right now.

-- Lars

On Tuesday, March 9, 2021, 10:54:16 AM PST, Kadir Ozdemir 
 wrote: 





Hi All,

This is a friendly reminder that our next tech talk meeting will be held at
9AM PST on April 01. If you like to present a topic for the next meeting,
please let us know by March 18. For more information about our tech talks,
and the slides and recording for the previous presentation, please visit
https://phoenix.apache.org/tech_talks.html.

I look forward to your suggestions for the next meeting.

Thanks,
Kadir

Re: Topic suggestions for April Tech Talk

[la...@apache.org]

In addition to the technical implementation of the PQS an interesting topic for 
the PQS would be how to scale it.
How many PQSs compared to the number of region servers? How to size the 
machine/VM? Etc.
I think just that could take more than 15 minutes. :)

On Thursday, March 18, 2021, 9:18:40 AM PDT, Josh Elser  
wrote: 





What I think we have now is..

* 15mins Hue
* 15mins on PQS and Python

I think a full hour might be too much, depends on amount of Q

On 3/17/21 6:24 PM, Kadir Ozdemir wrote:
> Josh,
> 
> I will be very interested in a discussion on PQS and would like to learn
> about the Python library and its integration with Hue.  Do you suggest to
> discuss all in one meeting?
> 
> Thanks,
> Kadir
> 
> On Wed, Mar 17, 2021 at 2:26 PM Josh Elser  wrote:
> 
>> While we try to figure out who will give it, I think I am safe to say we
>> can offer a discussion on PQS, the Phoenix Python library, and its
>> integration with Hue [1].
>>
>> Is this of interest?
>>
>> [1]
>> https://urldefense.com/v3/__https://gethue.com/__;!!DCbAVzZNrAf4!QOuUB00ICNpifpvXyDyti8dJ2abET-ZDa7qbcbPi11gGN_tl7bvbiT08g0J2zRqRUA$
>>
>> On 3/9/21 1:54 PM, Kadir Ozdemir wrote:
>>> Hi All,
>>>
>>> This is a friendly reminder that our next tech talk meeting will be held
>>> at 9AM PST on April 01. If you like to present a topic for the next
>>> meeting, please let us know by March 18. For more information about our
>>> tech talks, and the slides and recording for the previous presentation,
>>> please visit
>> https://urldefense.com/v3/__https://phoenix.apache.org/tech_talks.html__;!!DCbAVzZNrAf4!QOuUB00ICNpifpvXyDyti8dJ2abET-ZDa7qbcbPi11gGN_tl7bvbiT08g0I2wVK0pA$
>>
>>> <
>> https://urldefense.com/v3/__https://phoenix.apache.org/tech_talks.html__;!!DCbAVzZNrAf4!QOuUB00ICNpifpvXyDyti8dJ2abET-ZDa7qbcbPi11gGN_tl7bvbiT08g0I2wVK0pA$
>>> .
>>>
>>> I look forward to your suggestions for the next meeting.
>>>
>>> Thanks,
>>> Kadir
>>
> 

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC2

[la...@apache.org]

That test passes locally for me (with HBase 2.4.1).






On Wednesday, March 17, 2021, 1:28:45 PM PDT, Xinyi Yan  
wrote: 





Hi Geoffrey,

The ViewTTLIT doesn't have any test failures in the last 10 runs, see attached 
screenshot and link. Should we treat this as a test flapper instead of a 
blocker?



https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/5.1/test_results_analyzer/


Thanks,
Xinyi

On Wed, Mar 17, 2021 at 12:51 PM Geoffrey Jacoby  wrote:
> -0 (binding, will change to 1 or -1 based on whether this can be reproduced)
> 
> signatures and checksums: ok
> rat:check: ok
> mvn clean install -DskipTests: ok
> mvn verify: Test failures with ViewTTLIT
> 
> When run as part of the larger suite, ViewTTLIT failed with:
> 
> [ERROR] Tests run: 26, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 638.087 s <<< FAILURE! - in org.apache.phoenix.end2end.ViewTTLIT
> 
> [ERROR]
> testDeleteFromMultipleGlobalIndexes(org.apache.phoenix.end2end.ViewTTLIT)
> Time elapsed: 134.661 s  <<< FAILURE!
> 
> java.lang.AssertionError: Rows should exists before expiration
> 
> at org.junit.Assert.fail(Assert.java:89)
> 
> at org.junit.Assert.assertTrue(Assert.java:42)
> 
> at
> org.apache.phoenix.end2end.ViewTTLIT.validateExpiredRowsAreNotReturnedUsingCounts(ViewTTLIT.java:2350)
> 
> at org.apache.phoenix.end2end.ViewTTLIT.
> *testDeleteFromMultipleGlobalIndexes*(ViewTTLIT.java:2111)
> 
> And when run separately several times, it consistently failed with:
> 
> [*ERROR*] *Tests **run: 26*, *Failures: 1*, Errors: 0, Skipped: 0, Time
> elapsed: 725.558 s* <<< FAILURE!* - in org.apache.phoenix.end2end.
> *ViewTTLIT*
> 
> [*ERROR*]
> testDeleteWithOnlyTenantView(org.apache.phoenix.end2end.ViewTTLIT)  Time
> elapsed: 4.351 s  <<< FAILURE!
> 
> org.junit.ComparisonFailure: Values upserted and fetched do not match
> expected: but was:
> 
> at org.junit.Assert.assertEquals(Assert.java:117)
> 
> at
> org.apache.phoenix.end2end.ViewTTLIT.verifyRowsBeforeTTLExpiration(ViewTTLIT.java:2462)
> 
> at
> org.apache.phoenix.end2end.ViewTTLIT.validateExpiredRowsAreNotReturnedUsingData(ViewTTLIT.java:2390)
> 
> at
> org.apache.phoenix.end2end.ViewTTLIT.upsertDataAndRunValidations(ViewTTLIT.java:2333)
> 
> at org.apache.phoenix.end2end.ViewTTLIT.*testDeleteWithOnlyTenantView*
> (ViewTTLIT.java:1967)
> 
> 
> Geoffrey
> 
> 
> 
> On Tue, Mar 16, 2021 at 10:41 AM la...@apache.org  wrote:
> 
>> +1 (binding)
>>
>> Built from source, ran some test loads with a few million rows on a local
>> install.
>> Ran some choice tests.
>>
>> Nothing weird in the logs.
>>
>>
>> -- Lars
>>
>>
>>
>> On Tuesday, March 16, 2021, 8:28:43 AM PDT, Istvan Toth 
>> wrote:
>>
>>
>>
>>
>>
>> Please vote on this Apache phoenix release candidate,
>> phoenix-5.1.1RC2
>>
>> The VOTE will remain open for at least 72 hours.
>>
>> [ ] +1 Release this package as Apache phoenix 5.1.1
>> [ ] -1 Do not release this package because ...
>>
>> The tag to be voted on is 5.1.1RC2:
>>
>> https://github.com/apache/phoenix/tree/5.1.1RC2
>>
>> The release files, including signatures, digests, as well as CHANGES.md
>> and RELEASENOTES.md included in this RC can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC2/
>>
>> Maven artifacts are available in a staging repository at:
>>
>>   https://repository.apache.org/#stagingRepositories
>> (orgapachephoenix-1229)
>>
>> Artifacts were signed with the 0x794433C7 key which can be found in:
>>
>>   https://dist.apache.org/repos/dist/release/phoenix/KEYS
>>
>> To learn more about Apache phoenix, please see
>>
>>   http://phoenix.apache.org/
>>
>> Thanks,
>> Istvan
>>
> 

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC2

[la...@apache.org]

+1 (binding)

Built from source, ran some test loads with a few million rows on a local 
install.
Ran some choice tests.

Nothing weird in the logs.


-- Lars



On Tuesday, March 16, 2021, 8:28:43 AM PDT, Istvan Toth  
wrote: 





Please vote on this Apache phoenix release candidate,
phoenix-5.1.1RC2

The VOTE will remain open for at least 72 hours.

[ ] +1 Release this package as Apache phoenix 5.1.1
[ ] -1 Do not release this package because ...

The tag to be voted on is 5.1.1RC2:

https://github.com/apache/phoenix/tree/5.1.1RC2

The release files, including signatures, digests, as well as CHANGES.md
and RELEASENOTES.md included in this RC can be found at:

https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC2/

Maven artifacts are available in a staging repository at:

  https://repository.apache.org/#stagingRepositories
(orgapachephoenix-1229)

Artifacts were signed with the 0x794433C7 key which can be found in:

  https://dist.apache.org/repos/dist/release/phoenix/KEYS

To learn more about Apache phoenix, please see

  http://phoenix.apache.org/

Thanks,
Istvan

Re: [VOTE] Release of Apache Phoenix 5.1.1 RC1

[la...@apache.org]

Just came here to say the same.

While we're at it, let's also pull in PHOENIX-6409, as it bring better 
visibility into local index plans.

-- Lars

On Thursday, March 11, 2021, 11:41:52 PM PST, Viraj Jasani  
wrote: 





Thanks Istvan.
I believe we can get in PHOENIX-6385 with 5.1.1 since it is better perf
improvement even for count(*) queries. Anoop has fixed the issue in
HBASE-25644 and it will be included with upcoming HBase releases 2.4.2 and
2.3.5. However, good to resolve from Phoenix side too (change is trivial
but that way we will not have to wait for HBASE-25644 to land).

Are you fine with another RC (my bad, I was traveling for past 2 days and
still available intermittently only, else would have informed and raised PR
before this RC)?
Although the change can be backported to 4.x for better alignment, it
mainly impacts 5.x.


On Thu, 11 Mar 2021 at 5:18 PM, Istvan Toth  wrote:

> Please vote on this Apache phoenix release candidate,
> phoenix-5.1.1RC1
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache phoenix 5.1.1
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 5.1.1RC1:
>
>  https://github.com/apache/phoenix/tree/5.1.1RC1
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC1/
>
> Maven artifacts are available in a staging repository at:
>
>  https://repository.apache.org/#stagingRepositories
> (orgapachephoenix-1228)
>
> Artifacts were signed with the 0x794433C7 key which can be found in:
>
>  https://dist.apache.org/repos/dist/release/phoenix/KEYS
>
> To learn more about Apache phoenix, please see
>
>  http://phoenix.apache.org/
>
> Thanks,
> Istvan
>

Re: Release of Apache Phoenix 5.1.1 RC0

[la...@apache.org]

I just committed PHOENIX-6402

Let's have another attempt at an RC for 5.1.1.
(Assuming all tests continue to pass)

-- Lars


On Sunday, March 7, 2021, 3:30:46 PM PST, la...@apache.org  
wrote: 





I have a real fix ready in PHOENIX-6402. Let's wait for that and then do a 
5.1.1 release.






On Monday, March 1, 2021, 4:15:40 PM PST, la...@apache.org  
wrote: 





-1 (binding)

See https://issues.apache.org/jira/browse/PHOENIX-6400
I'm testing a bit more, but that looks wrong.

-- Lars



On Sunday, February 28, 2021, 7:32:09 PM PST, Xinyi Yan  
wrote: 





+1 (non-binding) based on the signature, checksum and mvn clean
install/verify.


On Sun, Feb 28, 2021 at 4:59 AM Viraj Jasani  wrote:

> +1 (non-binding)
>
> Tested against HBase 2.4.1 (compiled with Hadoop profile 3.0)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_171): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_171): ok
>  - mvn clean install  -DskipTests
> * CRUD testing locally: ok
> * Unit tests pass (1.8.0_171): failed
>  - mvn clean package  && mvn verify  -Dskip.embedded
>
>
> [ERROR] Failures:
> [ERROR]
>  IndexToolForNonTxGlobalIndexIT.testBuildSecondaryIndexAndScrutinize:421
> expected:<0> but was:<-1>
>
> [ERROR] Failures:
> [ERROR]
>  
>ViewTTLIT.testDeleteWithOnlyTenantView:1967->upsertDataAndRunValidations:2329->validateExpiredRowsAreNotReturnedUsingData:2390->verifyRowsBeforeTTLExpiration:2451
> Rows upserted and fetched do not match, upserted=5, fetched=4
> expected:<[00B0y01, 00B0y02, 00B0y03,
> 00B0y04, 00B0y05]> but was:<[00B0y01,
> 00B0y02, 00B0y03, 00B0y04]>
>
>
> Subsequent manual test runs went well:
>
> [INFO] Running org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
> [WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
> 1,010.964 s - in org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
>
> [INFO] Running org.apache.phoenix.end2end.ViewTTLIT
> [INFO] Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 772.233 s - in org.apache.phoenix.end2end.ViewTTLIT
>
>
>
> On 2021/02/27 00:51:06, Geoffrey Jacoby  wrote:
> > +1 (binding)
> >
> > Check signatures and checksums: OK
> > apache-rat:check for licenses : OK
> > mvn clean install -DskipTests: OK
> > mvn verify for new 2.4.1 support: OK after rerunning one crashed test
> >
> > Geoffrey
> >
> >
> > On Fri, Feb 26, 2021 at 6:04 AM Istvan Toth  wrote:
> >
> > > Please vote on this Apache phoenix release candidate,
> > > phoenix-5.1.1RC0
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache phoenix 5.1.1
> > > [ ] -1 Do not release this package because ...
> > >
> > > The tag to be voted on is 5.1.1RC0:
> > >
> > >  https://github.com/apache/phoenix/tree/5.1.1RC0
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC0/
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >  https://repository.apache.org/#stagingRepositories
> > > (orgapachephoenix-1215)
> > >
> > > Artifacts were signed with the 0x794433C7 key which can be found in:
> > >
> > >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> > >
> > > To learn more about Apache phoenix, please see
> > >
> > >  http://phoenix.apache.org/
> > >
> > > Thanks,
> > > Istvan
> > >
> >
>

Re: Release of Apache Phoenix 5.1.1 RC0

[la...@apache.org]

I have a real fix ready in PHOENIX-6402. Let's wait for that and then do a 
5.1.1 release.






On Monday, March 1, 2021, 4:15:40 PM PST, la...@apache.org  
wrote: 





-1 (binding)

See https://issues.apache.org/jira/browse/PHOENIX-6400
I'm testing a bit more, but that looks wrong.

-- Lars



On Sunday, February 28, 2021, 7:32:09 PM PST, Xinyi Yan  
wrote: 





+1 (non-binding) based on the signature, checksum and mvn clean
install/verify.


On Sun, Feb 28, 2021 at 4:59 AM Viraj Jasani  wrote:

> +1 (non-binding)
>
> Tested against HBase 2.4.1 (compiled with Hadoop profile 3.0)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_171): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_171): ok
>  - mvn clean install  -DskipTests
> * CRUD testing locally: ok
> * Unit tests pass (1.8.0_171): failed
>  - mvn clean package  && mvn verify  -Dskip.embedded
>
>
> [ERROR] Failures:
> [ERROR]
>  IndexToolForNonTxGlobalIndexIT.testBuildSecondaryIndexAndScrutinize:421
> expected:<0> but was:<-1>
>
> [ERROR] Failures:
> [ERROR]
>  
>ViewTTLIT.testDeleteWithOnlyTenantView:1967->upsertDataAndRunValidations:2329->validateExpiredRowsAreNotReturnedUsingData:2390->verifyRowsBeforeTTLExpiration:2451
> Rows upserted and fetched do not match, upserted=5, fetched=4
> expected:<[00B0y01, 00B0y02, 00B0y03,
> 00B0y04, 00B0y05]> but was:<[00B0y01,
> 00B0y02, 00B0y03, 00B0y04]>
>
>
> Subsequent manual test runs went well:
>
> [INFO] Running org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
> [WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
> 1,010.964 s - in org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
>
> [INFO] Running org.apache.phoenix.end2end.ViewTTLIT
> [INFO] Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 772.233 s - in org.apache.phoenix.end2end.ViewTTLIT
>
>
>
> On 2021/02/27 00:51:06, Geoffrey Jacoby  wrote:
> > +1 (binding)
> >
> > Check signatures and checksums: OK
> > apache-rat:check for licenses : OK
> > mvn clean install -DskipTests: OK
> > mvn verify for new 2.4.1 support: OK after rerunning one crashed test
> >
> > Geoffrey
> >
> >
> > On Fri, Feb 26, 2021 at 6:04 AM Istvan Toth  wrote:
> >
> > > Please vote on this Apache phoenix release candidate,
> > > phoenix-5.1.1RC0
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache phoenix 5.1.1
> > > [ ] -1 Do not release this package because ...
> > >
> > > The tag to be voted on is 5.1.1RC0:
> > >
> > >  https://github.com/apache/phoenix/tree/5.1.1RC0
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC0/
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >  https://repository.apache.org/#stagingRepositories
> > > (orgapachephoenix-1215)
> > >
> > > Artifacts were signed with the 0x794433C7 key which can be found in:
> > >
> > >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> > >
> > > To learn more about Apache phoenix, please see
> > >
> > >  http://phoenix.apache.org/
> > >
> > > Thanks,
> > > Istvan
> > >
> >
>

Re: Running tests locally

[la...@apache.org]

I see. Thanks. I thought at least the tests would work.

On Monday, March 1, 2021, 10:00:16 PM PST, Viraj Jasani  
wrote: 





Lars, IncompatibleClassChangeError is expected unless
HBase 2.x is built with Hadoop.profile 3.0 specifically.

https://github.com/apache/phoenix/blob/master/BUILDING.md


On 2021/03/02 02:03:03, "la...@apache.org"  wrote: 
> AHA...
> Looks like hadoop-ci is failing with the same exception, since Feb 18th, 
> looks like PHOENIX-6359 is the culprit.
> 
> So it's not me.
> 
> 
> On Monday, March 1, 2021, 5:46:51 PM PST, la...@apache.org  
> wrote: 
> 
> 
> 
> 
> 
> My apologies if this is dumb question - I've been working on other projects 
> for a while, now coming back to some Phoenix work.
> 
> I'm trying to simply run mvn test -Dtest=LocalIndexIT locally.
> 
> And the test always fails with:
> [ERROR] org.apache.phoenix.end2end.index.LocalIndexIT  Time elapsed: 39.412 s 
>  <<< ERROR!
> java.lang.RuntimeException: java.io.IOException: Shutting down
> at org.apache.phoenix.query.BaseTest.initMiniCluster(BaseTest.java:549)
> at org.apache.phoenix.query.BaseTest.setUpTestCluster(BaseTest.java:449)
> at 
> org.apache.phoenix.query.BaseTest.checkClusterInitialized(BaseTest.java:435)
> at org.apache.phoenix.query.BaseTest.setUpTestDriver(BaseTest.java:517)
> at 
> org.apache.phoenix.end2end.index.BaseLocalIndexIT.doSetup(BaseLocalIndexIT.java:65)
> 
> And in the logs I see this:
> 2021-03-01 17:37:56,572 ERROR [master/think:0:becomeActiveMaster] 
> org.slf4j.helpers.MarkerIgnoringBase(159): Failed to become active master
> java.lang.IncompatibleClassChangeError: Found interface 
> org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536)
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112)
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616)
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611)
> at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624)
> at 
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at 
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:662)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:848)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:551)
> at 
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:492)
> at 
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:161)
> at 
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:63)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296)
> at 
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:187)
> at 
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:207)
> at 
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:307)
> at 
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:827)
> at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2082)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:506)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 
> So it looks like it pulling the wrong version of Hadoop for running tests.
> Am I the only one seeing this?
> As I said my apologies if I am missing something. Has been a while.
> 
> Cheers.
> 
> -- Lars
> 

Re: Running tests locally

[la...@apache.org]

AHA...
Looks like hadoop-ci is failing with the same exception, since Feb 18th, looks 
like PHOENIX-6359 is the culprit.

So it's not me.


On Monday, March 1, 2021, 5:46:51 PM PST, la...@apache.org  
wrote: 





My apologies if this is dumb question - I've been working on other projects for 
a while, now coming back to some Phoenix work.

I'm trying to simply run mvn test -Dtest=LocalIndexIT locally.

And the test always fails with:
[ERROR] org.apache.phoenix.end2end.index.LocalIndexIT  Time elapsed: 39.412 s  
<<< ERROR!
java.lang.RuntimeException: java.io.IOException: Shutting down
at org.apache.phoenix.query.BaseTest.initMiniCluster(BaseTest.java:549)
at org.apache.phoenix.query.BaseTest.setUpTestCluster(BaseTest.java:449)
at org.apache.phoenix.query.BaseTest.checkClusterInitialized(BaseTest.java:435)
at org.apache.phoenix.query.BaseTest.setUpTestDriver(BaseTest.java:517)
at 
org.apache.phoenix.end2end.index.BaseLocalIndexIT.doSetup(BaseLocalIndexIT.java:65)

And in the logs I see this:
2021-03-01 17:37:56,572 ERROR [master/think:0:becomeActiveMaster] 
org.slf4j.helpers.MarkerIgnoringBase(159): Failed to become active master
java.lang.IncompatibleClassChangeError: Found interface 
org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624)
at 
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at 
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:662)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:848)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:551)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:492)
at 
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:161)
at 
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:63)
at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:187)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:207)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:307)
at 
org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:827)
at 
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2082)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:506)
at java.base/java.lang.Thread.run(Thread.java:834)

So it looks like it pulling the wrong version of Hadoop for running tests.
Am I the only one seeing this?
As I said my apologies if I am missing something. Has been a while.

Cheers.

-- Lars

Running tests locally

[la...@apache.org]

My apologies if this is dumb question - I've been working on other projects for 
a while, now coming back to some Phoenix work.

I'm trying to simply run mvn test -Dtest=LocalIndexIT locally.

And the test always fails with:
[ERROR] org.apache.phoenix.end2end.index.LocalIndexIT  Time elapsed: 39.412 s  
<<< ERROR!
java.lang.RuntimeException: java.io.IOException: Shutting down
at org.apache.phoenix.query.BaseTest.initMiniCluster(BaseTest.java:549)
at org.apache.phoenix.query.BaseTest.setUpTestCluster(BaseTest.java:449)
at org.apache.phoenix.query.BaseTest.checkClusterInitialized(BaseTest.java:435)
at org.apache.phoenix.query.BaseTest.setUpTestDriver(BaseTest.java:517)
at 
org.apache.phoenix.end2end.index.BaseLocalIndexIT.doSetup(BaseLocalIndexIT.java:65)

And in the logs I see this:
2021-03-01 17:37:56,572 ERROR [master/think:0:becomeActiveMaster] 
org.slf4j.helpers.MarkerIgnoringBase(159): Failed to become active master
java.lang.IncompatibleClassChangeError: Found interface 
org.apache.hadoop.hdfs.protocol.HdfsFileStatus, but class was expected
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:536)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$400(FanOutOneBlockAsyncDFSOutputHelper.java:112)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:616)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$8.doCall(FanOutOneBlockAsyncDFSOutputHelper.java:611)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createOutput(FanOutOneBlockAsyncDFSOutputHelper.java:624)
at 
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:53)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:180)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at 
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:662)
at 
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:848)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:551)
at 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:492)
at 
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:161)
at 
org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:63)
at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:296)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:187)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:207)
at 
org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:307)
at 
org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:827)
at 
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2082)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:506)
at java.base/java.lang.Thread.run(Thread.java:834)

So it looks like it pulling the wrong version of Hadoop for running tests.
Am I the only one seeing this?
As I said my apologies if I am missing something. Has been a while.

Cheers.

-- Lars

Re: Release of Apache Phoenix 5.1.1 RC0

[la...@apache.org]

-1 (binding)

See https://issues.apache.org/jira/browse/PHOENIX-6400
I'm testing a bit more, but that looks wrong.

-- Lars



On Sunday, February 28, 2021, 7:32:09 PM PST, Xinyi Yan  
wrote: 





+1 (non-binding) based on the signature, checksum and mvn clean
install/verify.


On Sun, Feb 28, 2021 at 4:59 AM Viraj Jasani  wrote:

> +1 (non-binding)
>
> Tested against HBase 2.4.1 (compiled with Hadoop profile 3.0)
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_171): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_171): ok
>  - mvn clean install  -DskipTests
> * CRUD testing locally: ok
> * Unit tests pass (1.8.0_171): failed
>  - mvn clean package  && mvn verify  -Dskip.embedded
>
>
> [ERROR] Failures:
> [ERROR]
>  IndexToolForNonTxGlobalIndexIT.testBuildSecondaryIndexAndScrutinize:421
> expected:<0> but was:<-1>
>
> [ERROR] Failures:
> [ERROR]
>  
>ViewTTLIT.testDeleteWithOnlyTenantView:1967->upsertDataAndRunValidations:2329->validateExpiredRowsAreNotReturnedUsingData:2390->verifyRowsBeforeTTLExpiration:2451
> Rows upserted and fetched do not match, upserted=5, fetched=4
> expected:<[00B0y01, 00B0y02, 00B0y03,
> 00B0y04, 00B0y05]> but was:<[00B0y01,
> 00B0y02, 00B0y03, 00B0y04]>
>
>
> Subsequent manual test runs went well:
>
> [INFO] Running org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
> [WARNING] Tests run: 72, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
> 1,010.964 s - in org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
>
> [INFO] Running org.apache.phoenix.end2end.ViewTTLIT
> [INFO] Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 772.233 s - in org.apache.phoenix.end2end.ViewTTLIT
>
>
>
> On 2021/02/27 00:51:06, Geoffrey Jacoby  wrote:
> > +1 (binding)
> >
> > Check signatures and checksums: OK
> > apache-rat:check for licenses : OK
> > mvn clean install -DskipTests: OK
> > mvn verify for new 2.4.1 support: OK after rerunning one crashed test
> >
> > Geoffrey
> >
> >
> > On Fri, Feb 26, 2021 at 6:04 AM Istvan Toth  wrote:
> >
> > > Please vote on this Apache phoenix release candidate,
> > > phoenix-5.1.1RC0
> > >
> > > The VOTE will remain open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache phoenix 5.1.1
> > > [ ] -1 Do not release this package because ...
> > >
> > > The tag to be voted on is 5.1.1RC0:
> > >
> > >  https://github.com/apache/phoenix/tree/5.1.1RC0
> > >
> > > The release files, including signatures, digests, as well as CHANGES.md
> > > and RELEASENOTES.md included in this RC can be found at:
> > >
> > >  https://dist.apache.org/repos/dist/dev/phoenix/phoenix-5.1.1RC0/
> > >
> > > Maven artifacts are available in a staging repository at:
> > >
> > >  https://repository.apache.org/#stagingRepositories
> > > (orgapachephoenix-1215)
> > >
> > > Artifacts were signed with the 0x794433C7 key which can be found in:
> > >
> > >  https://dist.apache.org/repos/dist/release/phoenix/KEYS
> > >
> > > To learn more about Apache phoenix, please see
> > >
> > >  http://phoenix.apache.org/
> > >
> > > Thanks,
> > > Istvan
> > >
> >
>

Re: [DISCUSS] Releasing Phoenix 5.1.1

[la...@apache.org]

Actually it was relatively easy to work around PHOENIX-6377 - at least with 
Trino (formerly Presto) where we discovered the problem in the first place.

That said, I'll hold off Phoenix 5.x support for Trino until after Phoenix 
5.1.1 as long as we can get a release out soon'ish.

 On Sunday, February 14, 2021, 5:04:08 AM PST, Istvan Toth  
wrote: 





Thanks Viraj.
Yes, if we can get HBase 2.4.1 support finished in a reasonable time, it
would be nice to include it in 5.1.1.

Istvan

On Sun, Feb 14, 2021 at 8:37 AM Viraj Jasani  wrote:

> Thanks Istvan, i agree to have 5.1.1 rolled out sooner due to PHOENIX-6377.
> I have also merged PHOENIX-6343 to 5.1 but it’s not any blocker or strong
> reason for another release as such.
>
> Moreover, i was thinking of supporting HBase 2.4.1 with Phoenix 5.1.1
> (PHOENIX-6359). Sounds good to you?
>
>
> On Sat, 13 Feb 2021 at 11:41 PM, Istvan Toth  wrote:
>
> > Hi!
> >
> > As PHOENIX-6377 is a major problem for projects that consume the
> > phoenix-client maven artifact, I think that we should release a patch
> > release with the fix sooner than later.
> > As soon as 4.16.0 is out, I plan to start the 5.1.1 release process from
> > the 5.1 branch.
> >
> > Let me know your thoughts, opinion, or ideas for other fixes that should
> go
> > into 5.1.1.
> >
> > regards
> > Istvan
> >
>

Re: [DISCUSS] Support incompatible patch release of HBase

[la...@apache.org]

FWIW, I agree.

As I'm experiencing with the Trino (formerly Presto) Phoenix connector, it is 
hard as is to support different versions of Phoenix (and HBase)...
To the point where I'm adding a second Phoenix connector to Trino just for 
Phoenix 5.1.0+ support (and Phoenix 5.0 cannot be supported at all).

Now this is just client support, so perhaps that's slightly easier, but 
definitely something to keep in mind as ecosystem integration will become, 
IMHO, more and more important.


That goes for Trino, Spark, for both direct support as well as snapshot support 
via Phoenix snapshot iterators. That last part is interesting since we now also 
need an HDFS client, all with the right versions (or the right stuff shaded in).

Sorry for going even more off tangent. Phoenix adoption is also dependent on 
how well it can work together with other BigData platforms.

-- Lars

On Wednesday, February 17, 2021, 1:44:40 AM PST, Istvan Toth 
 wrote: 





Thanks for your thoughtful response, Geoffrey.

I agree with all of your points for the most part.

Going a bit off tangent, and referencing to the EOL HBase support issue
thread, I personally think that if we want to increase
Phoenix adoption, we should build confidence in the Hbase community that
they can count on having a Phoenix that works on their HBase cluster
for a reasonable amount of time, preferably with updates.

This would include - again, in my personal opinion - not dropping support
for HBase releases in Phoenix patch releases as a policy.

Back in the other thread, Ankit argues that this would cause undue extra
work. IMO having more users and eventually contributors
would be well worth the extra effort that sticking to some predictable
policies would need.

In this case, we are fully equipped to handle the multiple 2.4.x versions,
Viraj's current patch needs just a few character changes,
and we can support both 2.4.0 and 2.4.1 in 5.1.1

Publishing binaries only the latest profiles, or for all profiles is a
matter of enumerating them in maven property.

Istvan


On Wed, Feb 17, 2021 at 7:34 AM Geoffrey Jacoby  wrote:

> Istvan,
>
> What makes the case tricky is that the broken class is IA.Private. HBase
> has set out compatibility guidelines about what is and isn't safe for
> external projects to depend upon, and in this particular case it looks like
> we were trying to do something using an IA.Private API that isn't safe to
> use externally. HBase is allowed to break Phoenix on a patch level when
> Phoenix depends on private APIs.
>
> One thing to follow-up on would be to work with the HBase community to
> allow us to replace our IA.Private calls with a new IA.LimitedPrivate or
> IA.Public API that does have compatibility guarantees. From looking at the
> Phoenix code, it looks like we're just trying to create StoreFiles as the
> output of a MapReduce job, which seems a reasonable thing for there to be
> HBase support for, in addition to HBase's own internal MapReduce formats.
> However, that new public or limited private API would only be usable with
> HBase 2.5 and up (since interface changes need a minor release.)
>
> I agree with either Andrew's kind offer to bring back temporary support of
> the old HStore methods in 2.4.2, or alternatively to only support 2.4.1 and
> up in Phoenix 5.1.1 (since 5.1 exists with 2.4.0 support and there's a
> workaround for PHOENIX-6377 for anyone who needs it). Would be good to
> avoid the extra compatibility profile this time, even if we eventually need
> more in the future for other reasons.
>
> Geoffrey
>
> On Tue, Feb 16, 2021 at 9:53 PM Istvan Toth 
> wrote:
>
> > Putting back the methods for 2.4.2 and blacklisting 2.4.1 could be a
> > solution for this specific case. (thanks for the offer)
> >
> > However similar past changes were more involved, where this wouldn't be
> an
> > option. The last two that I remember:
> > - multi-join support in 2.0, 2.1, 2.2
> > - raw filter support in 2.2
> >
> > I think that we should expect that these api breakages happen, and handle
> > these cases as the norm,
> > and have a policy for them.
> >
> > (Not that we shouldn't ask HBase not to break us in a patch level, if it
> is
> > just for the sake of refactoring)
> >
> > Istvan
> >
> > On Wed, Feb 17, 2021 at 5:49 AM Andrew Purtell  >
> > wrote:
> >
> > > If you opened a PR that puts the HStore methods back, marks them
> > > @Deprecated, and implements them by calling the new StoreUtil methods,
> I
> > > would approve it. Just for branch-2.4. Call it a courtesy for Phoenix.
> > >
> > > Assuming this gets released in 2.4.2, around the end of the month, then
> > > you could opt to blacklist 2.4.1 in your build rather than deal with
> it.
> > > Then this becomes an issue for a future 2.5 compat module.
> > >
> > >
> > > > On Feb 16, 2021, at 8:38 PM, Viraj Jasani 
> wrote:
> > > >
> > > > Thanks Andrew! That’s it, HBASE-25249 is the only refactoring that
> > broke
> > > > compat and since HStore is IA.Private, I could not 

Re: Roadmap to releasing Phoenix 5.1, PQS 6.0 and Connectors 6.0

[la...@apache.org]

Geoffrey did the forward port for consistent indexes.

I'll add PHOENIX-5712 to the list of blockers - we've gotten ourselves into a 
bind there with backwards compatibility,
and there's still not clear fix in sight as far as I can see. That same problem 
exists for4.16.

On the HBase front all innovation is going into the 2.x branches and we have 
currently no stable release out to support that
(I would not call 5.0 stable).

Anything else *really* blocking? There 42 jiras open against 4.16 and 64 
against 5.1.

-- Lars

On Monday, August 10, 2020, 12:59:26 PM PDT, Josh Elser  
wrote: 





Agreed with Istvan on the importance of the new secondary indexing.

Thanks for the super-clear list, Geoffrey. I'll spin out clones of each 
of the BLOCKERs you mention and tag them for 5.1.0 (not that they get lost).

Thanks you, Istvan, for your very thoughtful write-up. Looks like a 
great plan.

On 8/5/20 1:11 AM, Istvan Toth wrote:
> Hi!
> 
> Strongly Consistent Global Indexes is indeed the killer new feature in 5.1,
> and it's important to have it working as well as possible.
> I was only vaguely aware of the deficiencies in master, thanks for
> explaining it (and working on fixing them)
> 
> Thanks
> Istvan
> 
> On Tue, Aug 4, 2020 at 9:22 PM Geoffrey Jacoby  wrote:
> 
>> Thanks, Istvan, Richard, Josh and Rajeshbabu (and anyone else I missed :-)
>> ) for all your hard work getting master and the ancillary projects into a
>> releasable state.
>>
>> An additional task that I think should happen before 5.1 can be released is
>> getting the indexing code in master up to parity with 4.x. As you may know,
>> between 4.14 and the upcoming 4.16, a lot of work has been done to build a
>> new, self-repairing consistent secondary index framework for Phoenix. Most
>> of that work has been ported to the master branch, but there are still some
>> significant gaps.
>>
>> The biggest gap comes from the new indexing framework relying heavily on
>> Phoenix's SCN "lookback" feature to do point-in-time selects during index
>> creation and verification. SCN has in the past been an unreliable tool,
>> because HBase's flush and major compaction code cleans up expired versions
>> and removes chunks of prior history. To work around this, in 4.16 we've
>> introduced "max lookback age" in PHOENIX-5645, which allows operators to
>> configure a moving window during which compactions and flushes will not
>> purge versions for any history.
>>
>> PHOENIX-5645 doesn't exist in master, because the coprocessor changes in
>> HBase 2.0 made implementing the needed compaction hooks impossible.
>> HBASE-24321, released in HBase 2.3, makes it possible again, though only
>> for Phoenix builds using the 2.3 profile. (Since it adds to an interface,
>> HBase compatibility guidelines prevent HBASE-24321 from being backported to
>> 2.1 or 2.2.)
>>
>> So, for 5.1 I believe we need forward ports for :
>> PHOENIX-5881 (implementing max lookback age for 2.3) - BLOCKER
>> PHOENIX-5735 (IndexTool verification distinguishes between inconsistencies
>> before or after max lookback age) - BLOCKER
>> PHOENIX-5928 (Simplification / perf improvement for index builds)
>> PHOENIX-5969 (Bug fix for querying indexes with limit clauses) - BLOCKER
>> PHOENIX-5951 (Configurable failure logging for past-max-lookback rows)
>> PHOENIX-6058 (Better behavior on verification when max lookback is disabled
>> -- needed for HBase 2.1 and 2.2 profiles) - BLOCKER
>>
>> Kadir, Swaroopa, Abhishek, and Gokcen, please add in any that I missed. :-)
>>  Happy to discuss what is and isn't a blocker.
>>
>> I plan to do PHOENIX-5881 next week, and the rest depends on it and will
>> follow afterward.
>>
>> Thanks,
>>
>> Geoffrey
>>
>>
>> On Mon, Aug 3, 2020 at 7:24 AM Istvan Toth  wrote:
>>
>>> Hi!
>>>
>>> It's been more than two years since we've released 5.0.0, and almost as
>>> long since Connectors and PQS have been split from the main repo.
>>>
>>> I believe that we are now at the point where we've solved, or are close
>> to
>>> solving the issues that have prevented us from releasing a useful and
>>> relevant 5.1.0 , as well as making an actual releases of PQS and
>> Connectors
>>> that are usable with both 5.x and 4.x.
>>>
>>> The two major blockers that are still open are
>>>
>>>    - PHOENIX-6010 Create phoenix-thirdparty, and consume guava through it
>>>    - PHOENIX-5784 Phoenix-connectors doesn't work with phoenix master
>>> branch
>>>
>>> but I hope that we can wrap those up in the next few weeks.
>>>
>>> This is going to be a complex process, as we'll have to release new
>>> versions of ALL of our components. To recap, the affected projects, (and
>>> their dependencies):
>>>
>>>    - phoenix-thirdparty
>>>    - tephra
>>>    - omid (phoenix-thirdparty)
>>>    - phoenix (tehpra ?, omid, phoenix-thirdparty)
>>>    - PQS
>>>    - Connectors
>>>
>>> The 5.1 release is also a point where we can revisit the decision to
>>> support Tephra. We have inherited those projects because of low 

public shaming on slow tests :)

[la...@apache.org]

Hi All,

I won't have time to go through various tests again to speed them up as I did 
last year.

Instead let me at least publicly shame the slowest tests that take more than 10 
minutes to run:

1,620.111 org.apache.phoenix.end2end.InListIT
1,486.974 org.apache.phoenix.end2end.ParameterizedIndexUpgradeToolIT
1,282.425 org.apache.phoenix.end2end.index.MutableIndexIT
971.583 org.apache.phoenix.end2end.SystemCatalogCreationOnConnectionIT
799.175 
org.apache.phoenix.end2end.PermissionNSEnabledWithCustomAccessControllerIT
797.802 org.apache.phoenix.end2end.IndexToolForNonTxGlobalIndexIT
753.556 
org.apache.phoenix.end2end.PermissionNSDisabledWithCustomAccessControllerIT
728.345 org.apache.phoenix.end2end.ConcurrentMutationsExtendedIT
718.888 
org.apache.phoenix.end2end.index.GlobalMutableNonTxIndexWithLazyPostBatchWriteIT
712.515 org.apache.phoenix.end2end.index.GlobalMutableNonTxIndexIT
703.694 org.apache.phoenix.end2end.index.LocalIndexIT
679.57 org.apache.phoenix.end2end.CostBasedDecisionIT
678.27 org.apache.phoenix.end2end.BackwardCompatibilityIT
671.469 org.apache.phoenix.end2end.IntArithmeticIT
663.921 org.apache.phoenix.end2end.GroupByIT
652.939 org.apache.phoenix.end2end.PermissionNSEnabledIT
639.8 org.apache.phoenix.end2end.PermissionsCacheIT
626.862 org.apache.phoenix.end2end.index.GlobalIndexCheckerIT
619.055 org.apache.phoenix.end2end.CaseStatementIT
611.94 org.apache.phoenix.end2end.PermissionNSDisabledIT
604.01 org.apache.phoenix.end2end.InQueryIT
602.923 org.apache.phoenix.end2end.AlterTableIT
600.986 org.apache.phoenix.end2end.index.LocalMutableTxIndexIT


(If you own one of these, please consider what you are doing to all other 
Phoenix developers who have to wait for hours to get the results of a test run)

Cheers.

-- Lars

Re: No builds on Phoenix-Master jenkins since Feb 19th?

[la...@apache.org]

Thanks Istvan.

Just checked the builds. Looks like the 4.x builds have not passed a single 
time since May 20th.

I hope I am missing something... Otherwise this would be pretty frustrating. :)
(Since pleading doesn't appear to help, maybe we should automatically block all 
commits until all tests pass...?)

-- Lars


On Tuesday, April 21, 2020, 5:33:52 AM PDT, Istvan Toth  
wrote: 





I've deleted all but the four Phoenix jobs listed in the JIRA.

Re: Latest Phoenix built from 4.x prevent HBase from starting

[la...@apache.org]

And like the server jar the compat jar should be installed into Phoenix' top 
level directory.






On Friday, May 15, 2020, 5:51:12 PM PDT, Andrew Purtell  
wrote: 





Hit this today too. It's not enough to place phoenix-server.jar on the
classpath, now you also have to select the appropriate compatibility module
and add it too. At the least installation documentation should be updated.

On Fri, May 15, 2020 at 3:48 PM la...@apache.org  wrote:

> This is the exception I'm seeing.
>
> 2020-05-15 15:35:36,098 FATAL [RS_OPEN_PRIORITY_REGION-think:16201-1]
> regionserver.HRegionServer: ABORTING region server
> think,16201,1589581955446: The coprocessor
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver threw
> java.lang.NoClassDefFoundError:
> org/apache/phoenix/compat/hbase/CompatRpcControllerFactory
>
> And indeed CompatRpcControllerFactory is missing from phoenix-server.jar.
>
> Is anybody else seeing this or am I missing something?
>
> -- Lars

>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
  - A23, Crosstalk

Re: Latest Phoenix built from 4.x prevent HBase from starting

[la...@apache.org]

I can fix that by copying phoenix-hbase-compat-1.5.0-4.16.0-SNAPSHOT.jar from 
phoenix/lib to hbase/lib.
That is not intended I assume. There used to be just the Phoenix server jar 
needed.


On Friday, May 15, 2020, 3:55:32 PM PDT, la...@apache.org  
wrote: 





PHOENIX-5808 looks suspicious (I do like the change, it just may introduce this 
problem)

On Friday, May 15, 2020, 3:48:02 PM PDT, la...@apache.org  
wrote: 





This is the exception I'm seeing.

2020-05-15 15:35:36,098 FATAL [RS_OPEN_PRIORITY_REGION-think:16201-1] 
regionserver.HRegionServer: ABORTING region server think,16201,1589581955446: 
The coprocessor org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver 
threw java.lang.NoClassDefFoundError: 
org/apache/phoenix/compat/hbase/CompatRpcControllerFactory

And indeed CompatRpcControllerFactory is missing from phoenix-server.jar.

Is anybody else seeing this or am I missing something?

-- Lars

Re: Latest Phoenix built from 4.x prevent HBase from starting

[la...@apache.org]

PHOENIX-5808 looks suspicious (I do like the change, it just may introduce this 
problem)

On Friday, May 15, 2020, 3:48:02 PM PDT, la...@apache.org  
wrote: 





This is the exception I'm seeing.

2020-05-15 15:35:36,098 FATAL [RS_OPEN_PRIORITY_REGION-think:16201-1] 
regionserver.HRegionServer: ABORTING region server think,16201,1589581955446: 
The coprocessor org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver 
threw java.lang.NoClassDefFoundError: 
org/apache/phoenix/compat/hbase/CompatRpcControllerFactory

And indeed CompatRpcControllerFactory is missing from phoenix-server.jar.

Is anybody else seeing this or am I missing something?

-- Lars

Latest Phoenix built from 4.x prevent HBase from starting

[la...@apache.org]

This is the exception I'm seeing.

2020-05-15 15:35:36,098 FATAL [RS_OPEN_PRIORITY_REGION-think:16201-1] 
regionserver.HRegionServer: ABORTING region server think,16201,1589581955446: 
The coprocessor org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver 
threw java.lang.NoClassDefFoundError: 
org/apache/phoenix/compat/hbase/CompatRpcControllerFactory

And indeed CompatRpcControllerFactory is missing from phoenix-server.jar.

Is anybody else seeing this or am I missing something?

-- Lars

Re: No builds on Phoenix-Master jenkins since Feb 19th?

[la...@apache.org]

+1 on removing test that are no longer relevant... As long as we keep test 
coverage.






On Wednesday, April 8, 2020, 10:49:16 PM PDT, Istvan Toth  
wrote: 





Yes, the tests are (and have been for some time) in a horrible shape for
all branches and profiles.

I've spent a few days on it, and now at least we are at the point where the
test runs do not hang indefinitely, or get randomly killed, but
there are still a lot of flaky tests, and unexplained random failures.

Been too optimistic, I've just seen a recent 4.x run test hang. Fortunately
it's a spurious test that I'm about to move to the PQS repo in a few
minutes.

We do get a few successful test runs on each branch / profile combination,
but they are few and far between.

We do have a lot of old and no longer relevant jobs in Apache. I have
deactivated a few myself. I support removing them to reduce confusion.
I think that the only relevant ones are:

https://builds.apache.org/view/M-R/view/Phoenix/job/PreCommit-PHOENIX-Build/
https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-4.x-matrix/
https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master-matrix/
and possibly
https://builds.apache.org/view/M-R/view/Phoenix/job/PhoenixFindTestTimeout/
though I haven't looked into the last one.


István

On Thu, Apr 9, 2020 at 1:30 AM la...@apache.org  wrote:

> Thanks Josh.
>
> unfortunately
>
> https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master-matrix/
>
> has not passed a single time (at least for the past 15 runs)
>
> -- Lars
>
>
> On Wednesday, April 8, 2020, 2:14:47 PM PDT, Josh Elser 
> wrote:
>
>
>
>
> It looks like the Phoenix-Master[1] build is now defunct after Istvan's
> hbase profile work. New job over in [2] which is a matrix job for all of
> HBase 2.0, 2.1, and 2.2.
>
> I think the question is whether or not we want to keep the old job
> around? I think the answer is "no".
>
> [1] https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master/
> [2]
> https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master-matrix/
>
> On 4/8/20 2:00 PM, la...@apache.org wrote:
> > Just looking at the Phoenix Jenkins jobs I noticed that was no build on
> master since for 3 weeks.
> > Is that in purpose? There were clearly changes on the master branch
> since then.
> >
> > Cheers.
> >
> > -- Lars
> >
>

Re: No builds on Phoenix-Master jenkins since Feb 19th?

[la...@apache.org]

Thanks Josh.

unfortunately 

https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master-matrix/

has not passed a single time (at least for the past 15 runs)

-- Lars


On Wednesday, April 8, 2020, 2:14:47 PM PDT, Josh Elser  
wrote: 




It looks like the Phoenix-Master[1] build is now defunct after Istvan's 
hbase profile work. New job over in [2] which is a matrix job for all of 
HBase 2.0, 2.1, and 2.2.

I think the question is whether or not we want to keep the old job 
around? I think the answer is "no".

[1] https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master/
[2] 
https://builds.apache.org/view/M-R/view/Phoenix/job/Phoenix-master-matrix/

On 4/8/20 2:00 PM, la...@apache.org wrote:
> Just looking at the Phoenix Jenkins jobs I noticed that was no build on 
> master since for 3 weeks.
> Is that in purpose? There were clearly changes on the master branch since 
> then.
> 
> Cheers.
> 
> -- Lars
> 

No builds on Phoenix-Master jenkins since Feb 19th?

[la...@apache.org]

Just looking at the Phoenix Jenkins jobs I noticed that was no build on master 
since for 3 weeks.
Is that in purpose? There were clearly changes on the master branch since then.

Cheers.

-- Lars

Re: Committers please look at the Phoenix tests and fix your failures

[la...@apache.org]

 And I cannot stress enough how important this is for the project. As an 
example: We had the tests fail for just a few days, during that time we have 
had check-ins that broke other test; now it's quite hard to figure out which 
recent change broke the other tests.
We need the test suite *always* passing. It's impossible to maintain a stable 
code base the size of Phoenix otherwise.
-- Lars
On Tuesday, January 14, 2020, 10:04:12 AM PST, la...@apache.org 
 wrote:  
 
  I spent a lot of time making QA better. It can be better, but it's stable 
enough. There're now very little excuses. "Test failure seems unrelated" is not 
an excuse anymore.(4.x-HBase-1.3 has some issue where HBase can't seem to start 
a cluster reliably... but all others are pretty stable.)
After chatting with Andrew Purtell, one things I was going to offer is to 
simply revert any change that breaks a test. Period.I'd volunteer some of my 
time (hey, isn't that what a Chief Architect in a Fortune 100 company should 
do?!)
With their changes reverted, people will presumably start to care. :)If I hear 
no objects I'll start doing that a while.
Cheers.
-- Lars
    On Monday, January 13, 2020, 06:23:01 PM PST, Josh Elser 
 wrote:  
 
 How do we keep getting into this mess: unreliable QA, people ignoring 
QA, or something else?

On 1/12/20 9:24 PM, la...@apache.org wrote:
> ... Not much else to say here...
> The tests have been failing again for a while... I will NOT fix them again 
> this time! Sorry folks.
> 
> -- Lars
> 
> 
    

Re: Moving Phoenix master to Hbase 2.2

[la...@apache.org]

 Does somebody volunteer to take this up?
I can see whether I can a resource where I work, but it's highly uncertain.
It would need a bit of digging and design work to see how we would abstract the 
HBase interface in the most effective way.
As mentioned below, Tephra did a good job at this and could serve as an example 
here. (Not dinging OMID, OMID does most of it's work client side and doesn't 
need these abstractions.)
-- Lars

On Tuesday, January 14, 2020, 01:13:36 AM PST, István Tóth 
 wrote:  
 
 Yes, the HBase API signatures change between versions, so we need to
compile each compat module against a specific HBase.

Whether I can define an internal compatibility API that is switchable at
run (startup) time without a performance hit remains to be seen.

István

On Tue, Jan 14, 2020 at 3:21 AM Josh Elser  wrote:

> Agree that trying to wrangle branches is just too frustrating and
> error-prone.
>
> It would also be great if we could have a single Phoenix jar that works
> across HBase versions, but would not die on that hill :)
>
> On 12/20/19 5:04 AM, la...@apache.org wrote:
> >  I said _provided_ they can be isolated easily :) (I meant it in the
> sense of assuming it's easy).
> > As I said though, Tephra has a similar problem and they did a really
> good job isolating HBase versions. We can learn from them. Sometimes they
> isolate the change only, and sometimes the class needs to be copied, but
> even then it's the one class that is copied, not another branch that needs
> to be kept in sync.
> >
> > This may also drive the desperately necessary refactoring of Phoenix to
> make these things easier to isolate, or to reduce the copying to a minimum.
> And we'd need to think through testing carefully.
> >
> > The branch per Phoenix and HBase version is too complex, IMHO. And the
> complex branch to HBase version mapping that Istvan outlines below confirms
> that.
> >
> > We should all take a brief look at the Tephra solution and see whether
> we can apply that. (And since Tephra is part of the fold now, perhaps
> someone can help there...?)
> > Cheers.
> > -- Lars
> >
> >      On Thursday, December 19, 2019, 8:34:15 PM GMT+1, Geoffrey Jacoby <
> gjac...@gmail.com> wrote:
> >
> >  Lars,
> >
> > I'm curious why you say the differences are easily isolated -- many of
> the
> > core classes of Phoenix either directly inherit HBase classes or
> implement
> > HBase interfaces, and those can vary between minor versions. (See my
> above
> > example of a new coprocessor hook on BaseRegionObserver.)
> >
> > Geoffrey
> >
> > On Thu, Dec 19, 2019 at 10:54 AM la...@apache.org 
> wrote:
> >
> >>    Yep. The differences are pretty minimal - provided they can be
> isolated
> >> easily.
> >> Tephra might be a pretty good model. It supports various versions of
> HBase
> >> in a single branch and has similar issues as Phoenix (coprocessors,
> etc).
> >> -- Lars
> >>      On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser <
> >> els...@apache.org> wrote:
> >>
> >>    To clarify, you think that compat modules are better than that
> >> separate-branches model in 4.x?
> >>
> >> On 12/18/19 11:29 AM, la...@apache.org wrote:
> >>> This is really hard to follow.
> >>>
> >>> I think we should do the same with HBase dependencies in Phoenix that
> >> HBase does with Hadoop dependencies.
> >>>
> >>> That is:  We could have a maven module with the specific HBase version
> >> dependent code.
> >>> Btw. Tephra does the same... A module for HBase version specific code.
> >>> -- Lars
> >>>
> >>>        On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth <
> >> st...@apache.org> wrote:
> >>>
> >>>    What do you think about tying the minor releases to Hbase minor
> releases
> >>> (not necessarily one-to-one)
> >>>
> >>> for example (provided 5.1 is 2020H1)
> >>>
> >>> 5.0.0 -> HB 2.0
> >>> 5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
> >>> 5.1.x -> HB 2.2.x (treat as maintenance branch, no major new features)
> >>> 5.2.0 -> HB 2.3.0 (if released by that time)
> >>> 5.2.x -> HB 2.3.x (treat as maintenance branch, no major new features)
> >>> 5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
> >>> master -> latest released HBase version
> >>>
> >>> Alternatively, we could stick with the

Re: Committers please look at the Phoenix tests and fix your failures

[la...@apache.org]

 I spent a lot of time making QA better. It can be better, but it's stable 
enough. There're now very little excuses. "Test failure seems unrelated" is not 
an excuse anymore.(4.x-HBase-1.3 has some issue where HBase can't seem to start 
a cluster reliably... but all others are pretty stable.)
After chatting with Andrew Purtell, one things I was going to offer is to 
simply revert any change that breaks a test. Period.I'd volunteer some of my 
time (hey, isn't that what a Chief Architect in a Fortune 100 company should 
do?!)
With their changes reverted, people will presumably start to care. :)If I hear 
no objects I'll start doing that a while.
Cheers.
-- Lars
On Monday, January 13, 2020, 06:23:01 PM PST, Josh Elser 
 wrote:  
 
 How do we keep getting into this mess: unreliable QA, people ignoring 
QA, or something else?

On 1/12/20 9:24 PM, la...@apache.org wrote:
> ... Not much else to say here...
> The tests have been failing again for a while... I will NOT fix them again 
> this time! Sorry folks.
> 
> -- Lars
> 
> 
  

Committers please look at the Phoenix tests and fix your failures

[la...@apache.org]

... Not much else to say here...
The tests have been failing again for a while... I will NOT fix them again this 
time! Sorry folks.

-- Lars

Re: Python2 EOL

[la...@apache.org]

 Heh.
They used to be shell scripts and then we converted them to Python.Personally I 
was not a fan of that back then, but anyway.
In any case there's some work to do.

-- Lars

On Friday, January 10, 2020, 7:55:43 AM PST, Josh Elser  
wrote:  
 
 I think converting them to Bash is the right thing to do. We're not 
doing anything fancy.

On 1/9/20 5:10 PM, Andrew Purtell wrote:
> Some of the python scripts are glorified shell scripts and could be
> rewritten as such, such as the launch scripts for psql and sqlline and the
> pqs. I get that python is and was trendier than bash but sometimes the
> right tool for the job is the right tool for the job. Unlike python, bash
> has a very stable grammar.
> 
> On Thu, Jan 9, 2020 at 12:34 PM la...@apache.org  wrote:
> 
>> Hi all,
>>
>> python2 is officially EOL'd. No more changes, improvements, or fixes will
>> be done by the developers.
>> Some Linux distributions stopped shipping Python2.
>>
>> It turns out our scripts do not work with Python3, see: [PHOENIX-5656]
>> Make Phoenix scripts work with Python 3 - ASF JIRA.
>>
>> [PHOENIX-5656] Make Phoenix scripts work with Python 3 - ASF JIRA
>>
>> So what should we do?
>>
>> As outlined in the jira we have 3 options:
>>
>>      1. Do nothing. Phoenix will only work with EOL'd Python 2.
>>      2. try to make all the scripts work with Python 2 and 3. That's
>> actually not possible in cases, but we can get close... And it's a lot of
>> work and experimentation.
>>      3. Convert all scripts to Python 3. There's a tool (2to3) to do that
>> automatically. Phoenix will now _only_ work with Python 3.
>>
>> Option 2 is some work - some of it not trivial - that someone would need
>> to pick up. Perhaps we can maintain two versions of all scripts, figure out
>> the version of Python and the use right one?
>>
>> Let's discuss on the jira. I can't be only one interested in this :)
>>
>> Cheers.
>>
>> -- Lars
>>
> 
> 
  

Python2 EOL

[la...@apache.org]

Hi all,

python2 is officially EOL'd. No more changes, improvements, or fixes will be 
done by the developers.
Some Linux distributions stopped shipping Python2.

It turns out our scripts do not work with Python3, see: [PHOENIX-5656] Make 
Phoenix scripts work with Python 3 - ASF JIRA.

[PHOENIX-5656] Make Phoenix scripts work with Python 3 - ASF JIRA

So what should we do?

As outlined in the jira we have 3 options:

1. Do nothing. Phoenix will only work with EOL'd Python 2.
2. try to make all the scripts work with Python 2 and 3. That's actually 
not possible in cases, but we can get close... And it's a lot of work and 
experimentation.
3. Convert all scripts to Python 3. There's a tool (2to3) to do that 
automatically. Phoenix will now _only_ work with Python 3.

Option 2 is some work - some of it not trivial - that someone would need to 
pick up. Perhaps we can maintain two versions of all scripts, figure out the 
version of Python and the use right one?

Let's discuss on the jira. I can't be only one interested in this :)

Cheers.

-- Lars

Re: [ANNOUNCE] Apache Phoenix 4.15.0 released

[la...@apache.org]

 Yeah, we need to get better!

It's not generally possible to support newer clients with older servers. 
Phoenix might have added features that the old server cannot be expected to 
understand. So you in your app you'd have have to be incredibly careful not to 
use any new features.
Supporting more that one client version on the server is important, though.
Also, we need to keep in mind that there are three versions to consider (and 
this is was killed 4.15.0):
1. The server version
2. The client version
3. The version of the metadata.

We have been blissfully ignorant (to some extend at least) about #3. And that's 
why we need a metadata API. So that old clients are compatible with the new 
server code *and* the new metadata!

I think the expected mode of operations for larger installations would be to 
upgrade the server(s) and keep the older clients running for a while. Then when 
everything is deemed stable on the server switch the clients too.
Definitely a good discussion to have. I think it's time to do another Phoenux 
meetup and discuss these things...

-- Lars

On Saturday, December 21, 2019, 12:39:12 PM PST, Andrew Purtell 
 wrote:  
 
 One thing I would like to suggest is a change to the project compatibility 
policy for upgrade/downgrade to support the following:

Mixed client/server versions one minor release up or back. Going forward. So in 
the future eg client={ 4.14, 4.15, 4.16 } <-> server={ 4.14, 4.15, 4.16 }. This 
will allow for seamless rolling upgrades with rollback in all common scenarios. 

Then, consider an exception for some combinations of 4.15 and 5.1? I have 
tested this and these versions are data compatible (create data with 1.5/4.14, 
upgrade to 2.2/5.1) and at the HBase level the wire compatibility between 1 and 
2 is good enough for normal client operations. If the version check didn’t 
throw an exception I wonder if there would be any issue. 

For your consideration. 

> On Dec 21, 2019, at 2:40 AM, "la...@apache.org"  wrote:
> 
>  Yeah! Finally.
> Let's use this time to stabilize; especially the metadata management and 
> upgrade testing (and perhaps the HBase compatibility), so that we can have a 
> smooth 4.16.0 release.
> 
>    On Saturday, December 21, 2019, 1:19:34 AM GMT+1, Chinmay Kulkarni 
> wrote:  
> 
> Hello Everyone,
> 
> The Apache Phoenix team is pleased to announce the immediate availability
> of the 4.15.0 minor release. Apache Phoenix enables SQL-based OLTP and
> operational analytics for Apache Hadoop using Apache HBase as its backing
> store and providing integration with other projects in the Apache ecosystem
> such as Spark, Hive, Pig, Flume, and MapReduce.
> 
> This minor release has feature parity with supported HBase versions and
> includes the following improvements:
> - Support for multi-region SYSTEM.CATALOG
> - Omid integration with Phoenix
> - Orphan view tool
> - Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
> - 150+ bug fixes
> and much more
> 
> Download source and binaries here [1].
> 
> Thanks,
> Chinmay Kulkarni
> (on behalf of the Apache Phoenix team)
> 
> [1] http://phoenix.apache.org/download.html
> 
> -- 
> Chinmay Kulkarni  

Re: [ANNOUNCE] Apache Phoenix 4.15.0 released

[la...@apache.org]

 Yeah! Finally.
Let's use this time to stabilize; especially the metadata management and 
upgrade testing (and perhaps the HBase compatibility), so that we can have a 
smooth 4.16.0 release.

On Saturday, December 21, 2019, 1:19:34 AM GMT+1, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

The Apache Phoenix team is pleased to announce the immediate availability
of the 4.15.0 minor release. Apache Phoenix enables SQL-based OLTP and
operational analytics for Apache Hadoop using Apache HBase as its backing
store and providing integration with other projects in the Apache ecosystem
such as Spark, Hive, Pig, Flume, and MapReduce.

This minor release has feature parity with supported HBase versions and
includes the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

Download source and binaries here [1].

Thanks,
Chinmay Kulkarni
(on behalf of the Apache Phoenix team)

[1] http://phoenix.apache.org/download.html

-- 
Chinmay Kulkarni
  

Re: Moving Phoenix master to Hbase 2.2

[la...@apache.org]

 Yep that.
For the record... I do not think this is simple. But it is possible.

On Thursday, December 19, 2019, 8:37:37 PM GMT+1, Andrew Purtell 
 wrote:  
 
 I can't answer for Lars but whenever version incompatibilities come up
usually only a handful of files are impacted. In the last round, the
Phoenix access controller, a related file in the same directory, and the
RPC scheduler. If you cloned these into separate version specific maven
modules case by case as needed at each round the differences are fairly
small. On the other hand if you take a principled approach and abstract all
the things, it will be a huge effort and nobody realistically will want to
take it on.

On Thu, Dec 19, 2019 at 11:34 AM Geoffrey Jacoby  wrote:

> Lars,
>
> I'm curious why you say the differences are easily isolated -- many of the
> core classes of Phoenix either directly inherit HBase classes or implement
> HBase interfaces, and those can vary between minor versions. (See my above
> example of a new coprocessor hook on BaseRegionObserver.)
>
> Geoffrey
>
> On Thu, Dec 19, 2019 at 10:54 AM la...@apache.org 
> wrote:
>
> >  Yep. The differences are pretty minimal - provided they can be isolated
> > easily.
> > Tephra might be a pretty good model. It supports various versions of
> HBase
> > in a single branch and has similar issues as Phoenix (coprocessors, etc).
> > -- Lars
> >    On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser <
> > els...@apache.org> wrote:
> >
> >  To clarify, you think that compat modules are better than that
> > separate-branches model in 4.x?
> >
> > On 12/18/19 11:29 AM, la...@apache.org wrote:
> > > This is really hard to follow.
> > >
> > > I think we should do the same with HBase dependencies in Phoenix that
> > HBase does with Hadoop dependencies.
> > >
> > > That is:  We could have a maven module with the specific HBase version
> > dependent code.
> > > Btw. Tephra does the same... A module for HBase version specific code.
> > > -- Lars
> > >
> > >      On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth <
> > st...@apache.org> wrote:
> > >
> > >  What do you think about tying the minor releases to Hbase minor
> releases
> > > (not necessarily one-to-one)
> > >
> > > for example (provided 5.1 is 2020H1)
> > >
> > > 5.0.0 -> HB 2.0
> > > 5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
> > > 5.1.x -> HB 2.2.x (treat as maintenance branch, no major new features)
> > > 5.2.0 -> HB 2.3.0 (if released by that time)
> > > 5.2.x -> HB 2.3.x (treat as maintenance branch, no major new features)
> > > 5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
> > > master -> latest released HBase version
> > >
> > > Alternatively, we could stick with the same HBase version for patch
> > > releases that we used for the first minor release.
> > >
> > > This would limit the number of branches that we have to maintain in
> > > parallel, while providing maintenance branches for older releases, and
> > > timely-ish Phoenix releases.
> > >
> > > The drawback is that users of old HBase versions won't get the latest
> > > features, on the other hand they can expect more polish.
> > >
> > > Istvan
> > >
> > > On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby 
> > wrote:
> > >
> > >> Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x
> > >> compatibility with the 5.x branch going forward.
> > >>
> > >> Given how coupled Phoenix is to the implementation details of HBase
> > though,
> > >> I'm not sure trying to abstract those away to keep one Phoenix branch
> > per
> > >> HBase major version is practical, however. At the least, it would be
> > really
> > >> complex.
> > >>
> > >> For example, in the new year I plan to return to working on the change
> > data
> > >> capture and Phoenix-level replication features, both of which depend
> on
> > >> WALKey interface changes and a new RegionObserver coprocessor hook
> > >> introduced in HBASE-22622 and HBASE-22623. This was released in HBase
> > 1.5
> > >> and will be in the forthcoming HBase 2.3. While the HBase community is
> > >> discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium
> > term,
> > >> I don't see all pre-2.3 branch-2's getting deprecated anytime soon.
> > >>

Re: Moving Phoenix master to Hbase 2.2

[la...@apache.org]

 I said _provided_ they can be isolated easily :) (I meant it in the sense of 
assuming it's easy).
As I said though, Tephra has a similar problem and they did a really good job 
isolating HBase versions. We can learn from them. Sometimes they isolate the 
change only, and sometimes the class needs to be copied, but even then it's the 
one class that is copied, not another branch that needs to be kept in sync.

This may also drive the desperately necessary refactoring of Phoenix to make 
these things easier to isolate, or to reduce the copying to a minimum. And we'd 
need to think through testing carefully.

The branch per Phoenix and HBase version is too complex, IMHO. And the complex 
branch to HBase version mapping that Istvan outlines below confirms that.

We should all take a brief look at the Tephra solution and see whether we can 
apply that. (And since Tephra is part of the fold now, perhaps someone can help 
there...?)
Cheers.
-- Lars

On Thursday, December 19, 2019, 8:34:15 PM GMT+1, Geoffrey Jacoby 
 wrote:  
 
 Lars,

I'm curious why you say the differences are easily isolated -- many of the
core classes of Phoenix either directly inherit HBase classes or implement
HBase interfaces, and those can vary between minor versions. (See my above
example of a new coprocessor hook on BaseRegionObserver.)

Geoffrey

On Thu, Dec 19, 2019 at 10:54 AM la...@apache.org  wrote:

>  Yep. The differences are pretty minimal - provided they can be isolated
> easily.
> Tephra might be a pretty good model. It supports various versions of HBase
> in a single branch and has similar issues as Phoenix (coprocessors, etc).
> -- Lars
>    On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser <
> els...@apache.org> wrote:
>
>  To clarify, you think that compat modules are better than that
> separate-branches model in 4.x?
>
> On 12/18/19 11:29 AM, la...@apache.org wrote:
> > This is really hard to follow.
> >
> > I think we should do the same with HBase dependencies in Phoenix that
> HBase does with Hadoop dependencies.
> >
> > That is:  We could have a maven module with the specific HBase version
> dependent code.
> > Btw. Tephra does the same... A module for HBase version specific code.
> > -- Lars
> >
> >      On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth <
> st...@apache.org> wrote:
> >
> >  What do you think about tying the minor releases to Hbase minor releases
> > (not necessarily one-to-one)
> >
> > for example (provided 5.1 is 2020H1)
> >
> > 5.0.0 -> HB 2.0
> > 5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
> > 5.1.x -> HB 2.2.x (treat as maintenance branch, no major new features)
> > 5.2.0 -> HB 2.3.0 (if released by that time)
> > 5.2.x -> HB 2.3.x (treat as maintenance branch, no major new features)
> > 5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
> > master -> latest released HBase version
> >
> > Alternatively, we could stick with the same HBase version for patch
> > releases that we used for the first minor release.
> >
> > This would limit the number of branches that we have to maintain in
> > parallel, while providing maintenance branches for older releases, and
> > timely-ish Phoenix releases.
> >
> > The drawback is that users of old HBase versions won't get the latest
> > features, on the other hand they can expect more polish.
> >
> > Istvan
> >
> > On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby 
> wrote:
> >
> >> Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x
> >> compatibility with the 5.x branch going forward.
> >>
> >> Given how coupled Phoenix is to the implementation details of HBase
> though,
> >> I'm not sure trying to abstract those away to keep one Phoenix branch
> per
> >> HBase major version is practical, however. At the least, it would be
> really
> >> complex.
> >>
> >> For example, in the new year I plan to return to working on the change
> data
> >> capture and Phoenix-level replication features, both of which depend on
> >> WALKey interface changes and a new RegionObserver coprocessor hook
> >> introduced in HBASE-22622 and HBASE-22623. This was released in HBase
> 1.5
> >> and will be in the forthcoming HBase 2.3. While the HBase community is
> >> discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium
> term,
> >> I don't see all pre-2.3 branch-2's getting deprecated anytime soon.
> >>
> >> So there will be at least two significant features that can only exist
> in
> >> some but not all of our 4.x and 5.x branche

Re: Moving Phoenix master to Hbase 2.2

[la...@apache.org]

 Yep. The differences are pretty minimal - provided they can be isolated easily.
Tephra might be a pretty good model. It supports various versions of HBase in a 
single branch and has similar issues as Phoenix (coprocessors, etc).
-- Lars
On Thursday, December 19, 2019, 7:07:51 PM GMT+1, Josh Elser 
 wrote:  
 
 To clarify, you think that compat modules are better than that 
separate-branches model in 4.x?

On 12/18/19 11:29 AM, la...@apache.org wrote:
> This is really hard to follow.
> 
> I think we should do the same with HBase dependencies in Phoenix that HBase 
> does with Hadoop dependencies.
> 
> That is:  We could have a maven module with the specific HBase version 
> dependent code.
> Btw. Tephra does the same... A module for HBase version specific code.
> -- Lars
> 
>      On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth 
> wrote:
>  
>  What do you think about tying the minor releases to Hbase minor releases
> (not necessarily one-to-one)
> 
> for example (provided 5.1 is 2020H1)
> 
> 5.0.0 -> HB 2.0
> 5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
> 5.1.x -> HB 2.2.x (treat as maintenance branch, no major new features)
> 5.2.0 -> HB 2.3.0 (if released by that time)
> 5.2.x -> HB 2.3.x (treat as maintenance branch, no major new features)
> 5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
> master -> latest released HBase version
> 
> Alternatively, we could stick with the same HBase version for patch
> releases that we used for the first minor release.
> 
> This would limit the number of branches that we have to maintain in
> parallel, while providing maintenance branches for older releases, and
> timely-ish Phoenix releases.
> 
> The drawback is that users of old HBase versions won't get the latest
> features, on the other hand they can expect more polish.
> 
> Istvan
> 
> On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby  wrote:
> 
>> Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x
>> compatibility with the 5.x branch going forward.
>>
>> Given how coupled Phoenix is to the implementation details of HBase though,
>> I'm not sure trying to abstract those away to keep one Phoenix branch per
>> HBase major version is practical, however. At the least, it would be really
>> complex.
>>
>> For example, in the new year I plan to return to working on the change data
>> capture and Phoenix-level replication features, both of which depend on
>> WALKey interface changes and a new RegionObserver coprocessor hook
>> introduced in HBASE-22622 and HBASE-22623. This was released in HBase 1.5
>> and will be in the forthcoming HBase 2.3. While the HBase community is
>> discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium term,
>> I don't see all pre-2.3 branch-2's getting deprecated anytime soon.
>>
>> So there will be at least two significant features that can only exist in
>> some but not all of our 4.x and 5.x branches.
>>
>> Geoffrey
>>
>> On Thu, Dec 12, 2019 at 8:21 AM Josh Elser  wrote:
>>
>>> As much as possible, I'd like to avoid us getting into another situation
>>> with 5.x where we have multiple branches. My hope was/is that we can
>>> keep one Phoenix5 branch that works against an acceptable set of HBase
>>> branches.
>>>
>>> To me, that acceptable set of HBase branches is _a_ 2.1 and 2.2 release.
>>> I don't think we need to support all 2.1.x or 2.2.x, nor do I think we
>>> need to keep trying to maintain 2.0.x as it's already end of support by
>>> the HBase community.
>>>
>>> Thanks for updating your PR. I'll add this to my review queue.
>>>
>>> On 12/12/19 1:52 AM, Istvan Toth wrote:
>>>> Hi!
>>>>
>>>> I'd like to start a conversation about supporting HBase 2.2. in the
>>>> master branch.
>>>>
>>>> https://issues.apache.org/jira/browse/PHOENIX-5268 has a slightly out
>> of
>>>> date, but functional PR for HBase 2.2 support on master. (Please review
>>>> and comment if you have the time, I'll try to update the PR in the next
>>>> few days)
>>>>
>>>> The reason that it is not a straightforward decision to merge it is
>> that
>>>> applying that patch breaks compatibility with HBase 2.0.1, the current
>>>> base.
>>>>
>>>> I can see the following outcomes:
>>>>
>>>> - Do nothing
>>>> - Move master to HBase 2.2.2
>>>> - Fork master to Hbase-2.0 and Hbase-2.2 branches
>>>> - Build t

Re: Moving Phoenix master to Hbase 2.2

[la...@apache.org]

This is really hard to follow.

I think we should do the same with HBase dependencies in Phoenix that HBase 
does with Hadoop dependencies.

That is:  We could have a maven module with the specific HBase version 
dependent code.
Btw. Tephra does the same... A module for HBase version specific code.
-- Lars

On Tuesday, December 17, 2019, 10:00:31 AM GMT+1, Istvan Toth 
 wrote:  
 
 What do you think about tying the minor releases to Hbase minor releases
(not necessarily one-to-one)

for example (provided 5.1 is 2020H1)

5.0.0 -> HB 2.0
5.1.0 -> HB 2.2.2 (and whatever 2.1 is API compatible with it)
5.1.x -> HB 2.2.x (treat as maintenance branch, no major new features)
5.2.0 -> HB 2.3.0 (if released by that time)
5.2.x -> HB 2.3.x (treat as maintenance branch, no major new features)
5.3.0 -> HB 2.3.x (if there is no new major/minor Hbase release)
master -> latest released HBase version

Alternatively, we could stick with the same HBase version for patch
releases that we used for the first minor release.

This would limit the number of branches that we have to maintain in
parallel, while providing maintenance branches for older releases, and
timely-ish Phoenix releases.

The drawback is that users of old HBase versions won't get the latest
features, on the other hand they can expect more polish.

Istvan

On Thu, Dec 12, 2019 at 8:05 PM Geoffrey Jacoby  wrote:

> Since HBase 2.0 is EOM'ed, I'm +1 for not worrying about 2.0.x
> compatibility with the 5.x branch going forward.
>
> Given how coupled Phoenix is to the implementation details of HBase though,
> I'm not sure trying to abstract those away to keep one Phoenix branch per
> HBase major version is practical, however. At the least, it would be really
> complex.
>
> For example, in the new year I plan to return to working on the change data
> capture and Phoenix-level replication features, both of which depend on
> WALKey interface changes and a new RegionObserver coprocessor hook
> introduced in HBASE-22622 and HBASE-22623. This was released in HBase 1.5
> and will be in the forthcoming HBase 2.3. While the HBase community is
> discussing EOMing 1.3 right now, and maybe 1.4 will go in the medium term,
> I don't see all pre-2.3 branch-2's getting deprecated anytime soon.
>
> So there will be at least two significant features that can only exist in
> some but not all of our 4.x and 5.x branches.
>
> Geoffrey
>
> On Thu, Dec 12, 2019 at 8:21 AM Josh Elser  wrote:
>
> > As much as possible, I'd like to avoid us getting into another situation
> > with 5.x where we have multiple branches. My hope was/is that we can
> > keep one Phoenix5 branch that works against an acceptable set of HBase
> > branches.
> >
> > To me, that acceptable set of HBase branches is _a_ 2.1 and 2.2 release.
> > I don't think we need to support all 2.1.x or 2.2.x, nor do I think we
> > need to keep trying to maintain 2.0.x as it's already end of support by
> > the HBase community.
> >
> > Thanks for updating your PR. I'll add this to my review queue.
> >
> > On 12/12/19 1:52 AM, Istvan Toth wrote:
> > > Hi!
> > >
> > > I'd like to start a conversation about supporting HBase 2.2. in the
> > > master branch.
> > >
> > > https://issues.apache.org/jira/browse/PHOENIX-5268 has a slightly out
> of
> > > date, but functional PR for HBase 2.2 support on master. (Please review
> > > and comment if you have the time, I'll try to update the PR in the next
> > > few days)
> > >
> > > The reason that it is not a straightforward decision to merge it is
> that
> > > applying that patch breaks compatibility with HBase 2.0.1, the current
> > > base.
> > >
> > > I can see the following outcomes:
> > >
> > > - Do nothing
> > > - Move master to HBase 2.2.2
> > > - Fork master to Hbase-2.0 and Hbase-2.2 branches
> > > - Build time compatibility modules
> > > - Run time compatibility modules
> > > - Something that I haven't thought of
> > >
> > >
> > > Doing nothing is obviously not a long term solution, as the current
> > > master doesn't work with any of the currently supported HBase branches,
> > > but we may postpone the inevitable.
> > >
> > > Simply moving master to HBase 2.2 is the most attractive solution from
> a
> > > pure developer POV, but there may be other considerations.
> > >
> > > Having multiple masters for 2.0 and 2.2 is simple from a code
> > > perspective, but maintaining two branches is a non-trivial amount of
> > > additional work. (See the 4.x situation)
> > >
> > > Moving the HBase version dependent stuff into a separate module, and
> > > choosing at build time is not pretty from a code POV, but saves us the
> > > hassle of maintaining multiple branches, while maintaining
> compatibility
> > > with multiple  HBase versions, and can handle future API changes as
> well
> > > from a single branch. Doing something like this could have saved us the
> > > effort of maintaining three separate 4.x branches.
> > >
> > > I feel that since Phoenix is closely timed to HBase, and requires
> > 

Re: Multiple rows with same key

[la...@apache.org]

 You mean you see multiple rows with the same primary key? I have not see this 
- or heard about this before.

Could you post your schema?
-- Lars

On Tuesday, December 17, 2019, 7:42:13 PM GMT+1, Francesco Malerba 
 wrote:  
 
 Hi all, 
we are having some troubles with Phoenix 4.14 on top of Hbase 1.2, because some 
select are  returning duplicate rows with the same private. 
In our application, we make several upsert of rows with the same key since we 
have to keep synchronized an oracle and a phoenix table.

We started to notice this problem after the update of phoenix to version 4.14 
from version 4.7 and hbase from version 1.1 to version 1.2.

I suspect this issue could be somehow related to returning different versions 
of the same hbase row as different rows.
In order to look into this issue, my idea was to perform some scan directly on 
hbase, but i noticed that some bytes are appended after the first field of the 
PK and i'm not able to produce the equivalent scan of my select.

Finally my question are:
Is there a way to use the phoenix library to produce the equivalent bytes of my 
query to use in the Hbase Shell?

Could this issues be related to the update of phoenix, since the data were 
written using phoenix 4.7 and now we are reading them from a phoenix 4.14 
server ?

Any hint will be appreciated.
Thanks










  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC4

[la...@apache.org]

 +1 (binding)
Again... This time I did the following:
- Ran through upgrade and various test scripts using phoenix_sandbox.py (with 
PHOENIX-5617 applied). This started with 4.14, created some tables, views, and 
indexes, then with a 4.15 client, added/removed views, columns, and indexes, 
then with 4.14 accessed existing tables, views, and indexes, create more 
tables, views, and indexes, added/droped columns, etc, Then with 4.15 client 
again accessed those same tables, etc.

- With an actual install, inserted millions of rows, updated, queried, with 
views and indexes.
- All good.
I did see a strange exception in the server logs - see PHOENIX-5639, but this 
only happend in the sandbox and not with an actual install. So all fine.
-- Lars

On Tuesday, December 17, 2019, 6:33:07 AM GMT+1, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC4. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4 &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tar ball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc4/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc4/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc4/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc4/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc4/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc4/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12343162=12315120

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=65ee44ada9390ef46f6b2853e65067aa9ca1d598
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc4

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=f11afb2d609cbbfeaaee1cf981514be4c50a4bd0
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc4

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=a2adf5e572c5a4bcccee7f8ac43bad6b84293ec6
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc4

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team

-- 
Chinmay Kulkarni
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC3

[la...@apache.org]

 PMC, please test and vote on this :)


On Monday, December 9, 2019, 4:43:02 PM PST, la...@apache.org 
 wrote:  
 
  +1 (Binding)
Did the same tests as in RC2. :)
Cheers and crossing fingers.
-- Lars
    On Monday, December 9, 2019, 02:48:41 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC3. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tar ball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc3/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc3/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc3/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc3/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc3/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc3/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=2a1f74d5eab3da55aadb4dc1cf9673ebd2cfe2df
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc3

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=cbc92417aefc375b2afc3c601a1621a13379ed62
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc3

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=5204ac7f7f21fad0b495d02402303cb691d05036
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc3

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team

-- 
Chinmay Kulkarni
    

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC2

[la...@apache.org]

 This is the jira: https://issues.apache.org/jira/browse/PHOENIX-5607
There I listed a bunch of ideas and also what to test. Let's have a discussion 
there.The good news is that it is all fixable. I'm still excited about this - 
please do not read my previous email in any other way.
-- Lars
On Monday, December 9, 2019, 10:57:28 AM PST, Chinmay Kulkarni 
 wrote:  
 
 Thanks Lars, I agree that we need backwards compatibility tests. The past
couple of RC's for 4.15 have been rejected mainly due to backwards
compatibility issues. Good to mark this as a blocker for 4.16.

On Sat, Dec 7, 2019 at 10:39 AM la...@apache.org  wrote:

>  I filed a blocker bug for 4.16.0 to improve backward compatibility and
> tests for the same.And I wow to veto any 4.16 release without that in
> place. We've almost reached the point at which Phoenix becomes
> unreleasable, this has to stop.
>
> For 4.15.0 we have no chance but to release without that.
> However, there are some bigger features waiting to be committed soon (View
> TTL, et al). I will also veto those until we have 4.15.0 out, so that we do
> not destabilize 4.15.0 further. (Or we can create a set of 4.15 branches
> now and control commits there.)
>
> It pains me a bit that as a community we seem to fail following well known
> software engineering principles (always releasable code base, agile
> principles, frequent releases, comprehensive test suite, etc, etc).
>
> Cheers.
>
> -- Lars
>
>    On Friday, December 6, 2019, 1:57:50 PM PST, la...@apache.org <
> la...@apache.org> wrote:
>
>  Sigh. Agreed.
> I change my vote to -1 (binding) as well.
>
>    On Friday, December 6, 2019, 1:51:35 PM PST, Chinmay Kulkarni <
> chinmayskulka...@gmail.com> wrote:
>
>  Thanks for the votes everyone. Thanks for finding this bug Geoffrey, great
> debugging!
> Unfortunately this looks like a blocker and I change my vote to -1
> (binding).
>
> We will be back with an RC3 once this bug gets resolved.
>
> On Fri, Dec 6, 2019 at 12:33 PM Geoffrey Jacoby 
> wrote:
>
> > -1 (binding)
> >
> > - Built from source (OK)
> > - Verified signatures and checksums (OK)
> > - Ran mvn clean verify (OK)
> > - Tried to run all my use case's DDL with a 4.14 client and a 4.15 server
> > whose system tables weren't upgraded yet. It failed on an ALTER TABLE ADD
> > COLUMN for lack of SYSTEM.CHILD_LINK when doing child view lookup.
> >
> > Filed PHOENIX-5605. At the least, if the community decides this is an
> > acceptable limitation, there should be a clear error message saying that
> > this isn't allowed. But this seems to me to break our backward
> > compatibility guarantees.
> >
> > Geoffrey
> >
> > On Fri, Dec 6, 2019 at 11:17 AM Nitesh Maheshwari
> >  wrote:
> >
> > > +1 (Non-binding)
> > >
> > > Checked the following:
> > >
> > >    - Built from source
> > >    - Verified signatures
> > >    - Verified checksums
> > >    - Some sanity testing
> > >      - Verified schema, table, views, index creation
> > >      - Looked at some queries and their EXPLAIN PLAN output
> > >
> > > Thanks,
> > > Nitesh
> > >
> > > On Thu, Dec 5, 2019 at 8:41 PM Neha Gupta 
> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - Verified checksums : OK
> > > > - Verified signatures : OK
> > > > - Built from source (4.15.0-HBase-1.3) : OK
> > > > - mvn verify : OK
> > > > - Did some basic DDL, upserts, deletes, querying, and everything
> seems
> > > > fine.
> > > > - Created indexes and views queried them and dropped them, everything
> > > seems
> > > > fine.
> > > >
> > > >
> > > > On Thu, Dec 5, 2019 at 7:47 PM Daniel Wong
> > > >  wrote:
> > > >
> > > > >  +1 (non-binding)
> > > > > I did the following:
> > > > > New install on hbase latest 1.3 version on local hbase.
> > > > > Ran several queries and checked explain plan for right
> scans/filters
> > > > > including full table scans, range scans, point lookups, skip scan
> > > > filters,
> > > > > server side filters.
> > > > >
> > > > > On Thu, Dec 5, 2019 at 6:40 PM Chinmay Kulkarni <
> > > > > chinmayskulka...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1 (Binding)
> > > > > >
> > > > > > - Built from source (4.15.0-HBa

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC3

[la...@apache.org]

 +1 (Binding)
Did the same tests as in RC2. :)
Cheers and crossing fingers.
-- Lars
On Monday, December 9, 2019, 02:48:41 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC3. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tar ball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc3/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc3/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc3/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc3/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc3/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc3/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=2a1f74d5eab3da55aadb4dc1cf9673ebd2cfe2df
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc3

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=cbc92417aefc375b2afc3c601a1621a13379ed62
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc3

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=5204ac7f7f21fad0b495d02402303cb691d05036
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc3

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team

-- 
Chinmay Kulkarni
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC2

[la...@apache.org]

 I filed a blocker bug for 4.16.0 to improve backward compatibility and tests 
for the same.And I wow to veto any 4.16 release without that in place. We've 
almost reached the point at which Phoenix becomes unreleasable, this has to 
stop. 

For 4.15.0 we have no chance but to release without that.
However, there are some bigger features waiting to be committed soon (View TTL, 
et al). I will also veto those until we have 4.15.0 out, so that we do not 
destabilize 4.15.0 further. (Or we can create a set of 4.15 branches now and 
control commits there.)

It pains me a bit that as a community we seem to fail following well known 
software engineering principles (always releasable code base, agile principles, 
frequent releases, comprehensive test suite, etc, etc).

Cheers.

-- Lars

On Friday, December 6, 2019, 1:57:50 PM PST, la...@apache.org 
 wrote:  
 
  Sigh. Agreed.
I change my vote to -1 (binding) as well.

    On Friday, December 6, 2019, 1:51:35 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Thanks for the votes everyone. Thanks for finding this bug Geoffrey, great
debugging!
Unfortunately this looks like a blocker and I change my vote to -1
(binding).

We will be back with an RC3 once this bug gets resolved.

On Fri, Dec 6, 2019 at 12:33 PM Geoffrey Jacoby  wrote:

> -1 (binding)
>
> - Built from source (OK)
> - Verified signatures and checksums (OK)
> - Ran mvn clean verify (OK)
> - Tried to run all my use case's DDL with a 4.14 client and a 4.15 server
> whose system tables weren't upgraded yet. It failed on an ALTER TABLE ADD
> COLUMN for lack of SYSTEM.CHILD_LINK when doing child view lookup.
>
> Filed PHOENIX-5605. At the least, if the community decides this is an
> acceptable limitation, there should be a clear error message saying that
> this isn't allowed. But this seems to me to break our backward
> compatibility guarantees.
>
> Geoffrey
>
> On Fri, Dec 6, 2019 at 11:17 AM Nitesh Maheshwari
>  wrote:
>
> > +1 (Non-binding)
> >
> > Checked the following:
> >
> >    - Built from source
> >    - Verified signatures
> >    - Verified checksums
> >    - Some sanity testing
> >      - Verified schema, table, views, index creation
> >      - Looked at some queries and their EXPLAIN PLAN output
> >
> > Thanks,
> > Nitesh
> >
> > On Thu, Dec 5, 2019 at 8:41 PM Neha Gupta  wrote:
> >
> > > +1 (non-binding)
> > >
> > > - Verified checksums : OK
> > > - Verified signatures : OK
> > > - Built from source (4.15.0-HBase-1.3) : OK
> > > - mvn verify : OK
> > > - Did some basic DDL, upserts, deletes, querying, and everything seems
> > > fine.
> > > - Created indexes and views queried them and dropped them, everything
> > seems
> > > fine.
> > >
> > >
> > > On Thu, Dec 5, 2019 at 7:47 PM Daniel Wong
> > >  wrote:
> > >
> > > >  +1 (non-binding)
> > > > I did the following:
> > > > New install on hbase latest 1.3 version on local hbase.
> > > > Ran several queries and checked explain plan for right scans/filters
> > > > including full table scans, range scans, point lookups, skip scan
> > > filters,
> > > > server side filters.
> > > >
> > > > On Thu, Dec 5, 2019 at 6:40 PM Chinmay Kulkarni <
> > > > chinmayskulka...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 (Binding)
> > > > >
> > > > > - Built from source (4.15.0-HBase-1.3) : OK
> > > > > - mvn verify : OK
> > > > > - Did some basic DDL, upserts, deletes, querying, and everything
> > looked
> > > > > fine : OK
> > > > > - Verified checksums : OK
> > > > > - Verified signatures : OK
> > > > > - mvn clean apache-rat:check : OK
> > > > >
> > > > > On Thu, Dec 5, 2019 at 5:24 PM xinyi yan 
> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Tested locally on my environment, I created tables, views, index,
> > and
> > > > > > upsert data in different clients(4.15 and 4.14). All behaviors
> are
> > > > > expected
> > > > > > this time.
> > > > > >
> > > > > > Xinyi Yan
> > > > > >
> > > > > > On Thu, Dec 5, 2019 at 1:02 PM Thomas D'Silva <
> twdsi...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > T

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC2

[la...@apache.org]

 Sigh. Agreed.
I change my vote to -1 (binding) as well.

On Friday, December 6, 2019, 1:51:35 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Thanks for the votes everyone. Thanks for finding this bug Geoffrey, great
debugging!
Unfortunately this looks like a blocker and I change my vote to -1
(binding).

We will be back with an RC3 once this bug gets resolved.

On Fri, Dec 6, 2019 at 12:33 PM Geoffrey Jacoby  wrote:

> -1 (binding)
>
> - Built from source (OK)
> - Verified signatures and checksums (OK)
> - Ran mvn clean verify (OK)
> - Tried to run all my use case's DDL with a 4.14 client and a 4.15 server
> whose system tables weren't upgraded yet. It failed on an ALTER TABLE ADD
> COLUMN for lack of SYSTEM.CHILD_LINK when doing child view lookup.
>
> Filed PHOENIX-5605. At the least, if the community decides this is an
> acceptable limitation, there should be a clear error message saying that
> this isn't allowed. But this seems to me to break our backward
> compatibility guarantees.
>
> Geoffrey
>
> On Fri, Dec 6, 2019 at 11:17 AM Nitesh Maheshwari
>  wrote:
>
> > +1 (Non-binding)
> >
> > Checked the following:
> >
> >    - Built from source
> >    - Verified signatures
> >    - Verified checksums
> >    - Some sanity testing
> >      - Verified schema, table, views, index creation
> >      - Looked at some queries and their EXPLAIN PLAN output
> >
> > Thanks,
> > Nitesh
> >
> > On Thu, Dec 5, 2019 at 8:41 PM Neha Gupta  wrote:
> >
> > > +1 (non-binding)
> > >
> > > - Verified checksums : OK
> > > - Verified signatures : OK
> > > - Built from source (4.15.0-HBase-1.3) : OK
> > > - mvn verify : OK
> > > - Did some basic DDL, upserts, deletes, querying, and everything seems
> > > fine.
> > > - Created indexes and views queried them and dropped them, everything
> > seems
> > > fine.
> > >
> > >
> > > On Thu, Dec 5, 2019 at 7:47 PM Daniel Wong
> > >  wrote:
> > >
> > > >  +1 (non-binding)
> > > > I did the following:
> > > > New install on hbase latest 1.3 version on local hbase.
> > > > Ran several queries and checked explain plan for right scans/filters
> > > > including full table scans, range scans, point lookups, skip scan
> > > filters,
> > > > server side filters.
> > > >
> > > > On Thu, Dec 5, 2019 at 6:40 PM Chinmay Kulkarni <
> > > > chinmayskulka...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 (Binding)
> > > > >
> > > > > - Built from source (4.15.0-HBase-1.3) : OK
> > > > > - mvn verify : OK
> > > > > - Did some basic DDL, upserts, deletes, querying, and everything
> > looked
> > > > > fine : OK
> > > > > - Verified checksums : OK
> > > > > - Verified signatures : OK
> > > > > - mvn clean apache-rat:check : OK
> > > > >
> > > > > On Thu, Dec 5, 2019 at 5:24 PM xinyi yan 
> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Tested locally on my environment, I created tables, views, index,
> > and
> > > > > > upsert data in different clients(4.15 and 4.14). All behaviors
> are
> > > > > expected
> > > > > > this time.
> > > > > >
> > > > > > Xinyi Yan
> > > > > >
> > > > > > On Thu, Dec 5, 2019 at 1:02 PM Thomas D'Silva <
> twdsi...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Tested with standalone local hbase (version 1.3.5). Create a
> > table
> > > > > with a
> > > > > > > few indexes and ran some queries.
> > > > > > > Everything looks good.
> > > > > > >
> > > > > > > On Thu, Dec 5, 2019 at 11:35 AM Chinmay Kulkarni <
> > > > > > > chinmayskulka...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > A correction:
> > > > > > > >
> > > > > > > > Please use the CODE SIGNING KEY: "702918B6"
> > > > > > > >
> > > > > > > > Sorry for the inconvenience.
> > > > > > > >
> > > &g

Re: [ACTION REQUIRED] Omid & Tephra Migration to Phoenix LDAP

[la...@apache.org]

 That leads me to a general question:

(1) Will OMID and TEPHRA continue to be usable standalone without Phoenix? In 
that case we should keep the OMID and TEPHRA projects in Jira.

Or (2) do we envision to tightly integrate those into the Phoenix and have them 
only work in the Phoenix contexts?

I strongly feel that option #1 is the one to go for, and we should discuss and 
have an opinion.
In fact as I outlined in earlier messages, I think there are more parts of 
Phoenix that are useful by themselves and should be separated out, such as the 
type encoders and the key-building, as well as the high performance interface 
into HBase via coprocessors... among others.
-- Lars
   On Tuesday, December 3, 2019, 9:56:24 AM PST, Josh Elser  
wrote:  
 
 Andreas, Gokul, Terence, and Yoni: You four should be all set as Phoenix 
committers now. Please be sure that you're subscribed to the Phoenix dev 
list for now (we'll split out new mailing lists as the need arises).

I'll give INFRA the go-ahead to start tearing down the Omid and Tephra 
resources to move them under Phoenix.

For those who notice this at a later date, please include me in the 
"To:" line so that I don't miss your request to retain your privileges.

- Josh

On 11/26/19 1:25 PM, Josh Elser wrote:
> Hi,
> 
> As a part of the exit of the Incubator, we need to consolidate the Omid 
> and Tephra LDAP groups (the system that controls podling membership, 
> committer privileges, etc) with the Phoenix LDAP group.
> 
> For all of those Omid and Tephra podling members who intend to continue 
> to contribute to Omid or Tephra *and* are not already Phoenix committer 
> or PMC, please reply to this email (on dev@phoenix) with your ASF ID and 
> your intent to continue to contribute.
> 
> I know a holiday in the US is coming up, but please try to respond by 
> Monday (2019/12/02) COB. As close as we can get to the full set of 
> committers from Omid & Tephra, the easier you will all be making my 
> life. If someone does "miss" this message, please contact the Phoenix 
> PMC and we'll add you to the appropriate LDAP group at a later point 
> (your status does not expire, as per the ASF norms).
> 
> Thanks in advance.
> 
> - Josh (VP Phoenix)
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC2

[la...@apache.org]

 +1 (binding)
I did the following:
- built from source
- verified tar ball
- ran with tip of HBase branch-1
- created tables, global and local ndexes, views, and indexes on views with 
both 4.15.0 and older client
- upserted data alternately with a 4.15.0 and older client
- queried while upserting with 4.15.0 and older client
- caused some splits while upserting or querying
- Nothing weird in the logs.


-- Lars

On Monday, December 2, 2019, 9:46:07 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC2. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tar ball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc2/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc2/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc2/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc2/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc2/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc2/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=1f85c13a0935a0705f69b0122aa75ab8d6a634f2
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc2

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=e776bb0bc677b8f56571b06130a8fbf080b0a985
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc2

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=81ec7d22937cf3f83a4cb2be35d0ce5974dd1fec
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc2

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team

-- 
Chinmay Kulkarni
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC1

[la...@apache.org]

 Yeah, we need to sink RC1 as well. To be complete, here's my updated vote on 
RC1: -1 (binding)

The good news is that I fixed PHOENIX-5584, so we can spin an RC2 immediately.
-- Lars

On Monday, November 25, 2019, 10:23:27 AM PST, Chinmay Kulkarni 
 wrote:  
 
 After some more discussions on PHOENIX-5584, it’s confirmed that there is
definitely a backwards compatibility problem. I will spin up an RC2 after
we resolve the issue.

Thanks,
Chinmay

On Mon, Nov 25, 2019 at 7:03 AM Josh Elser  wrote:

> (sorry for piling on the vote thread)
>
> I would think that allowing rollback should be the _default_ case,
> especially when we know that there is an incompatibility.
>
> On 11/23/19 11:27 AM, Chinmay Kulkarni wrote:
> > Copying my comment from PHOENIX-5584:
> >
> > "On second thought, I actually think this is by design due to the change
> > introduced by PHOENIX-4893 and PHOENIX-4765 since we only continue
> writing
> > the parent table column metadata while creating a view in case
> > "phoenix.allow.system.catalog.rollback" is true. If this config is false
> > (which is the default case), this is not done.
> > I have verified that when the above config is true, this issue does not
> > exist and old clients are able to query the view properly."
> >
> > Seems like this may not be an issue after all. Thomas, what do you think?
> >
> >
> > On Fri, Nov 22, 2019 at 4:51 PM Geoffrey Jacoby 
> wrote:
> >
> >> Great find, Xinyi, thanks for testing!
> >>
> >> Geoffrey
> >>
> >> On Fri, Nov 22, 2019 at 4:39 PM Chinmay Kulkarni <
> >> chinmayskulka...@gmail.com>
> >> wrote:
> >>
> >>> I change my vote to -1 (binding). The issue that Xinyi pointed out is a
> >>> backwards compatibility blocker.
> >>> Also, I do not see the same bug with a 4.14 client and 4.13 client,
> which
> >>> means this seems to have been introduced in the 4.15 code.
> >>> I will get PHOENIX-5584 fixed and spin up a new RC. Thanks for finding
> >> this
> >>> bug Xinyi!
> >>>
> >>> On Fri, Nov 22, 2019 at 4:21 PM xinyi yan  wrote:
> >>>
> >>>> -1
> >>>>
> >>>> I found a bug that older client can't get right view metadata from
> 4.15
> >>>> server when 4.15 client created a table and view. I also created JIRA
> >>>> https://issues.apache.org/jira/browse/PHOENIX-5584 for this. In
> short,
> >>> you
> >>>> can reproduce the bug as following:
> >>>>
> >>>> 1. start the sandbox at 4.15.
> >>>>
> >>>> 2. created table and view at 4.15 client.
> >>>> 3. query `SELECT * FROM VIEW` at 4.14 client.
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Xinyi Yan
> >>>>
> >>>> On Fri, Nov 22, 2019 at 11:52 AM Neha Gupta 
> >>> wrote:
> >>>>
> >>>>> +1 (Non-Binding)
> >>>>> - Built from source (4.15.0-HBase-1.3)
> >>>>> - verified all IT tests
> >>>>> - Did some basic DDL, upserts, deletes, querying, and everything
> >> looks
> >>>>> good.
> >>>>> - Added some index and views, Upserts to view, querying and
> >> everything
> >>>>> looks good.
> >>>>>
> >>>>> On Thu, Nov 21, 2019 at 1:44 PM Vincent Poon  >>>
> >>>>> wrote:
> >>>>>
> >>>>>> +1 (Binding)
> >>>>>>
> >>>>>> Did some sanity testing with old client and new server.  Creating
> >>>> tables,
> >>>>>> global/local indexes, etc.  all seemed fine.
> >>>>>>
> >>>>>> On Thu, Nov 21, 2019 at 12:58 PM la...@apache.org <
> >> la...@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>  I agree that PHOENIX-5529 and PHOENIX-5580 are longstanding
> >> issues
> >>>> and
> >>>>>>> not blockers for 4.15.0. My +1 stands.
> >>>>>>> Other PCMs, please vote so that we can (finally) get a 4.15.0
> >>> release
> >>>>>> out.
> >>>>>>> Cheers.
> >>>>>>> -- Lars
> >>>>>>>
> >>>>>>>      On Wednesday, November 20, 2019, 7:23:01 PM PST, Chinmay
> >>>> Kulkarni <

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC1

[la...@apache.org]

 I agree that PHOENIX-5529 and PHOENIX-5580 are longstanding issues and not 
blockers for 4.15.0. My +1 stands.
Other PCMs, please vote so that we can (finally) get a 4.15.0 release out.
Cheers.
-- Lars

On Wednesday, November 20, 2019, 7:23:01 PM PST, Chinmay Kulkarni 
 wrote:  
 
 +1 (Binding)

- Built from source (4.15.0-HBase-1.3) : OK
- mvn verify : OK
- Did some basic DDL, upserts, deletes, querying, and everything looked
fine : OK
(Note, I found PHOENIX-5529
<https://issues.apache.org/jira/browse/PHOENIX-5529> and PHOENIX-5580
<https://issues.apache.org/jira/browse/PHOENIX-5580> during my testing, but
these bugs are also present in 4.14.x and hence not blockers for 4.15.0)
- Verified checksums : OK
- Verified signatures : OK
- mvn clean apache-rat:check : OK

On Wed, Nov 20, 2019 at 10:34 AM Chinmay Kulkarni <
chinmayskulka...@gmail.com> wrote:

> *REMINDER - [VOTE] Release of Apache Phoenix 4.15.0 RC1*
>
> Just sending out a friendly reminder to everyone to please vote on Apache
> Phoenix 4.15.0 RC1 (thanks Lars).
>
> On Tue, Nov 19, 2019 at 2:49 PM la...@apache.org  wrote:
>
>>  I responded yesterday, but somehow didn't appear to go through. Trying
>> again...
>>
>> +1 (Binding)
>>
>> - built from source (4.x-HBase-1.5) with latest branch-1 HBase (also
>> built from source)
>> - loaded a few million rows
>> - tried with local indexes
>> - issued upserts and deletes while splitting in HBase.
>> - checked the test suite on Jenkins :)
>> - all good
>> - nothing undue in the logs
>>
>> -- Lars
>>
>>    On Monday, November 18, 2019, 02:57:59 PM PST, Chinmay Kulkarni <
>> chinmayskulka...@gmail.com> wrote:
>>
>>  Hello Everyone,
>>
>> This is a call for a vote on Apache Phoenix 4.15.0 RC1. This is the next
>> minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
>> 1.5. The release includes both a source-only release and a convenience
>> binary release for each supported HBase version.
>>
>> This release has feature parity with supported HBase versions and includes
>> the following improvements:
>> - Support for multi-region SYSTEM.CATALOG
>> - Omid integration with Phoenix
>> - Orphan view tool
>> - Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
>> - 150+ bug fixes
>> and much more
>>
>> The source tarball, including signatures, digests, etc can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc1/src/
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc1/src/
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc1/src/
>>
>> The binary artifacts can be found at:
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc1/bin/
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc1/bin/
>>
>> https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc1/bin/
>>
>> For a complete list of changes, see:
>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162
>>
>> Artifacts are signed with my "CODE SIGNING KEY":
>> 7C5FC713DE4C59D7
>>
>> KEYS file available here:
>> https://dist.apache.org/repos/dist/dev/phoenix/KEYS
>>
>> The hash and tag to be voted upon:
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=a9491b4339ceef1c09922c752147fd97068039cd
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc1
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=f95b1937af1a525c2bf49e0a50bf73fce09a5314
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc1
>>
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=241a3284cf9128a08d0db49d07c2cfecceeada06
>>
>> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc1
>>
>> Vote will be open for at least 72 hours. Please vote:
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and reason why)
>>
>> Thanks,
>> The Apache Phoenix Team
>>
>> --
>> Chinmay Kulkarni
>>
>
> --
> Chinmay Kulkarni
>


-- 
Chinmay Kulkarni
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC1

[la...@apache.org]

 I responded yesterday, but somehow didn't appear to go through. Trying again...

+1 (Binding)

- built from source (4.x-HBase-1.5) with latest branch-1 HBase (also built from 
source)
- loaded a few million rows
- tried with local indexes
- issued upserts and deletes while splitting in HBase.
- checked the test suite on Jenkins :)
- all good
- nothing undue in the logs

-- Lars

On Monday, November 18, 2019, 02:57:59 PM PST, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC1. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tarball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc1/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc1/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc1/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc1/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc1/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc1/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=a9491b4339ceef1c09922c752147fd97068039cd
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc1

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=f95b1937af1a525c2bf49e0a50bf73fce09a5314
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc1

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=241a3284cf9128a08d0db49d07c2cfecceeada06
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc1

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team

-- 
Chinmay Kulkarni
  

Re: Nested Record

[la...@apache.org]

 The slightly longer answer is that HBase is very well equipped to handle this 
case.Row for inner/nested relationships can be stored inline with the parent 
relation, by extending the key by one more part indicating the identity of 
inner rows.The trick is to teach Phoenix to understand this - such as executing 
joins for such 1:many relationships as single scans, etc.
There's an old Jira: PHOENIX-150... Perhaps try to warm that one up again?
-- Lars
On Sunday, November 10, 2019, 02:53:07 AM PST, James Taylor 
 wrote:  
 
 Hi Charles,
Phoenix doesn’t support nested data. I suggest you take a look at other
open source projects such as Apache Drill or Presto.
Thanks,
James

On Sat, Nov 9, 2019 at 8:16 AM Charles Wiese  wrote:

> I was very excited about the use case of creating Patient Records with 1:M
> records nested (such as Diagnosis, Procedures etc).  We have done this in
> MongoDB and could use a SQL driver to get the nested “rows”.  I understand
> that the idea of nested “rows” ate actually columns with “column_name:id”
> for each “row value”.  Seems a bit odd compare to a document / JSON
> layout, but okay.  However, how does one use a SQL driver for reporting and
> “unnesting” this data?
>
> Is that even possible … I want to “select count(nested_procedures),
> patient_type from patient group by patient_type)”….. in with Mongo driver
> it is a virtual table like "select count(patient_procedures
> \.nested_procedures), patient_type from patient, patient_procedures group
> by patient_type)”
>
> Is this possible?  

Re: [VOTE] Accept Tephra and Omid podlings as Phoenix sub-projects

[la...@apache.org]

 +1

On Wednesday, October 30, 2019, 8:27:36 AM PDT, Josh Elser 
 wrote:  
 
 Hi,

As was previously discussed[1][2], there is a motion to "adopt" the 
Tephra and Omid podlings as sub-projects to Apache Phoenix. A 
sub-project is a distinct software project from some top-level project 
(TLP) but operates under the guidance of a TLP (e.g. the TLP's PMC)

Per the Incubator guidelines[3], we need to have a formal vote to adopt 
them. While we still need details from these two podlings, I believe we 
can have a VOTE now to keep things moving.

Actions:
* Phoenix will incorporate Omid and Tephra as sub-projects and they will 
continue to function under the Apache Phoenix PMC guidance.
* Any current podling member may request to be added as a Phoenix 
committer. Podling members would be expected to demonstrate the normal 
level of commitment to grow from a committer to a PMC member.

Stating what I see as an implicit decision (but to alleviate any 
possible confusion): all new community members will be expected to 
function in the way that Phoenix currently does today (e.g 
review-then-commit). Future Omid and Tephra development would happen in 
the same way that Phoenix development happens today.

Please vote:

+1: to accept Omid and Tephra as Phoenix sub-projects and to allow any 
PPMC to become Phoenix committers.

-1/-0/+0: No and why..

Here is my +1 (binding)

This vote will be open for at least 72hrs (2019/11/02 1600 UTC).


[1] 
https://lists.apache.org/thread.html/ec00615cbbb4225885e3776f1e8fd071600a9c50f35769f453b8a779@%3Cdev.phoenix.apache.org%3E
[2] 
https://lists.apache.org/thread.html/692a030a27067c20b9228602af502199cd4d80eb0aa8ed6461ebe1ee@%3Cgeneral.incubator.apache.org%3E
[3] 
https://incubator.apache.org/guides/graduation.html#graduating_to_a_subproject
  

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC0

[la...@apache.org]

 Hmm... I cannot reproduce this. I built from the tip of the 4.x-HBase-1.5 
branch.
I wiped HDFS and ZK. Then I ran HBase-1.4.10 with Phoenix-4.14.3, create a 
table, index, and view in Phoenix.Then I switched the HBase and Phoenix 
versions to 1.5.0 and 4.15.0 (tip of branch as above).Did not connect with the 
4.15 client, and then used the 4.13.3 client to describe tables, select from 
the table, etc. Just works.
So in my setup PHOENIX-5103 seems to work as expected, just PHOENIX-5533 is an 
issue.
Vincent could you double check on PHOENIX-5103 and post the precise step you 
took.
Cheers.

-- Lars

I do get the NPE when trying to create an index.

On Friday, October 18, 2019, 6:41:34 PM PDT, Chinmay Kulkarni 
 wrote:  
 
 Thanks for finding the backwards compatibility issues Vincent.
I also see a similar NPE when creating a view on top of a table with a 4.14
client.
I will re-open PHOENIX-5103 to solve the issue with dropping tables.

Opened PHOENIX-5533 <https://issues.apache.org/jira/browse/PHOENIX-5533> and
re-opened PHOENIX-5103
<https://issues.apache.org/jira/browse/PHOENIX-5103> which
are blockers, so -1 on RC0.
I will commit the fixes and spin up a new RC.

On Fri, Oct 18, 2019 at 4:48 PM Vincent Poon  wrote:

> Yea, I should have mentioned:
> This was built from source on the 1.3 branch, using the hash Chinmay
> provided: d41cc
> This was before any 4.15 client connected (i.e. metadata was not upgraded)
> So, this is a backwards compatibility issue.
>
> On Fri, Oct 18, 2019 at 4:25 PM la...@apache.org  wrote:
>
> >  Did you use one of the provided tarballs (in that case, which one)? Or
> > build from source (in that case, which branch?).
> > Also was that before or after the first 4.15 client connected with the
> > cluster (i.e. was the meta data upgraded or not)?
> > That would be a blocker, indeed. :(
> > -- Lars
> >    On Friday, October 18, 2019, 04:09:11 PM PDT, Vincent Poon <
> > vincentp...@apache.org> wrote:
> >
> >  I'm also not able to create an index with the older client:
> >
> > 2019-10-18 16:06:26,569 ERROR
> > [RpcServer.FifoWFPBQ.default.handler=255,queue=21,port=16201]
> > coprocessor.MetaDataEndpointImpl: createTable failed
> > java.lang.NullPointerException
> > at
> >
> >
> org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1758)
> > at
> >
> >
> org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:17218)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:8270)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2207)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2189)
> > at
> >
> >
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35076)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2373)
> > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> > at
> > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> > at
> > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> >
> > On Fri, Oct 18, 2019 at 3:21 PM Vincent Poon 
> > wrote:
> >
> > > -1
> > > I still see PHOENIX-5103 still cropping up for "DROP TABLE" when using
> > the
> > > 4.14 client against 4.15 server.  However, it seems "CREATE TABLE"
> works
> > > fine.
> > >
> > > Caused by: org.apache.hadoop.hbase.TableNotFoundException:
> > > org.apache.hadoop.hbase.TableNotFoundException: Table
> 'SYSTEM:CHILD_LINK'
> > > was not found, got: SYSTEM:CATALOG.
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1323)
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1195)
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(CoprocessorHConnection.java:41)
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:303)
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> > > at
> > >
> >
> org.apache.hadoop.hbase.client.ScannerCallab

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC0

[la...@apache.org]

 Did you use one of the provided tarballs (in that case, which one)? Or build 
from source (in that case, which branch?).
Also was that before or after the first 4.15 client connected with the cluster 
(i.e. was the meta data upgraded or not)?
That would be a blocker, indeed. :(
-- Lars
On Friday, October 18, 2019, 04:09:11 PM PDT, Vincent Poon 
 wrote:  
 
 I'm also not able to create an index with the older client:

2019-10-18 16:06:26,569 ERROR
[RpcServer.FifoWFPBQ.default.handler=255,queue=21,port=16201]
coprocessor.MetaDataEndpointImpl: createTable failed
java.lang.NullPointerException
at
org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1758)
at
org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:17218)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:8270)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2207)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2189)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35076)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2373)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)

On Fri, Oct 18, 2019 at 3:21 PM Vincent Poon  wrote:

> -1
> I still see PHOENIX-5103 still cropping up for "DROP TABLE" when using the
> 4.14 client against 4.15 server.  However, it seems "CREATE TABLE" works
> fine.
>
> Caused by: org.apache.hadoop.hbase.TableNotFoundException:
> org.apache.hadoop.hbase.TableNotFoundException: Table 'SYSTEM:CHILD_LINK'
> was not found, got: SYSTEM:CATALOG.
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1323)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1195)
> at
> org.apache.hadoop.hbase.client.CoprocessorHConnection.locateRegion(CoprocessorHConnection.java:41)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:303)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:314)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:289)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:164)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:159)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:803)
> at
> org.apache.hadoop.hbase.client.HTableWrapper.getScanner(HTableWrapper.java:235)
> at org.apache.phoenix.util.ViewUtil.hasChildViews(ViewUtil.java:163)
> at
> org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doDropTable(MetaDataEndpointImpl.java:2303)
> at
> org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:2153)
>
>
> On Fri, Oct 18, 2019 at 2:36 PM Chinmay Kulkarni <
> chinmayskulka...@gmail.com> wrote:
>
>> *REMINDER - [VOTE] Release of Apache Phoenix 4.15.0 RC0*
>>
>> Just sending out a friendly reminder to everyone to please vote on Apache
>> Phoenix 4.15.0 RC0 (thanks Lars).
>>
>> On Thu, Oct 17, 2019 at 3:28 PM la...@apache.org 
>> wrote:
>>
>> >  +1 (binding)
>> > - Built from source
>> > - Went through a simple 4.14 to 4.15 upgrade- Loaded a few dozens of
>> > millions of rows.- Tried global and local indexes.- Tried HBase 1.5.0-
>> > Nothing weird in the logs.
>> > Looks good.
>> > On a related note: I'd really love us to go back to a monthly release
>> > train. That would lead to smaller and more stable releases.Let's
>> schedule
>> > 4.15.1 (and perhaps 5.1.1...?) in a month from now. (If Chinmay is
>> tired of
>> > it, I'll volunteer to be RM :))
>> >
>> > -- Lars
>> >
>> >    On Thursday, October 17, 2019, 3:05:32 PM PDT, Chinmay Kulkarni <
>> > chinmayskulka...@gmail.com> wrote:
>> >
>> >  Hello Everyone,
>> >
>> > This is a call for a vote on Apache Phoenix 4.15.0 RC0. This is the next
>

Re: [VOTE] Release of Apache Phoenix 4.15.0 RC0

[la...@apache.org]

 +1 (binding)
- Built from source
- Went through a simple 4.14 to 4.15 upgrade- Loaded a few dozens of millions 
of rows.- Tried global and local indexes.- Tried HBase 1.5.0- Nothing weird in 
the logs.
Looks good.
On a related note: I'd really love us to go back to a monthly release train. 
That would lead to smaller and more stable releases.Let's schedule 4.15.1 (and 
perhaps 5.1.1...?) in a month from now. (If Chinmay is tired of it, I'll 
volunteer to be RM :))

-- Lars

On Thursday, October 17, 2019, 3:05:32 PM PDT, Chinmay Kulkarni 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.15.0 RC0. This is the next
minor release of Phoenix 4, compatible with Apache HBase 1.3, 1.4, &
1.5. The release includes both a source-only release and a convenience
binary release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
the following improvements:
- Support for multi-region SYSTEM.CATALOG
- Omid integration with Phoenix
- Orphan view tool
- Separation of the Phoenix-Connectors and Phoenix-Queryserver projects
- 150+ bug fixes
and much more

The source tarball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc0/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc0/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc0/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.3-rc0/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.4-rc0/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.15.0-HBase-1.5-rc0/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12343162

Artifacts are signed with my "CODE SIGNING KEY":
7C5FC713DE4C59D7

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=d41cc51d0b593bba4bc70b6edc95afb807fe8899
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.3-rc0

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=403607fa969d9cc760572f065e0b41027c8d584c
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.4-rc0

https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=c533eed363aedfdb3fb774d6ff510b287a9660ad
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;h=refs/tags/v4.15.0-HBase-1.5-rc0

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team
-- 
Chinmay Kulkarni
  

Re: Jenkins runs failing. Could not resolve host: gitbox.apache.org?

[la...@apache.org]

 Seems to be happening in H35 only. Removed it from the build rotation (for 
master for now)

On Thursday, July 18, 2019, 8:42:03 AM HST, la...@apache.org 
 wrote:  
 
 Does anybody know what could cause this? Failed at least the last 10 master 
builds (which I have deleted). I'll leave one failed build up.
  

Jenkins runs failing. Could not resolve host: gitbox.apache.org?

[la...@apache.org]

Does anybody know what could cause this? Failed at least the last 10 master 
builds (which I have deleted). I'll leave one failed build up.

Re: Volunteering to be RM for 4.15.0/5.1.0

[la...@apache.org]

 +1

On Monday, July 8, 2019, 12:57:27 PM HST, Thomas D'Silva 
 wrote:  
 
 I don't think we need to call 4.15/5.1 an alpha or beta release. Lets just
follow our usual process and release  4.15 and 5.1,
(after ensuring all the ITs pass and any manual testing that needs to be
done). If we need to fix bugs we can have a patch release.


On Mon, Jul 8, 2019 at 11:57 AM Chinmay Kulkarni 
wrote:

> Hi everyone,
>
> I would like to be RM for 4.15.0/5.1.0. There are about 18
> <
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%204.15.0
> >
> open issues for 4.15.0 and 22
> <
> https://issues.apache.org/jira/browse/PHOENIX-5386?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%205.1.0
> >
> for 5.1.0. Thanks a lot to Lars for doing the work required for getting a
> stable and quicker-running test suite and for getting the release efforts
> started!
>
> I'm working on some blockers and will be moving any new feature requests
> against 4.15/5.1 to 4.15.1 and 5.1.1 respectively.
>
> As per discussions on a previous email thread, let's aim to release a
> SNAPSHOT for people to test with and ensure that the test suite keeps
> passing without significant performance degradation.
>
> Thanks,
> Chinmay
> --
> Chinmay Kulkarni
>
  

Re: 4.15.0 and 5.1.0 releases

[la...@apache.org]

 Great!

Unless I hear otherwise I will steer people away from new features on the Jira 
- delay them to 4.15.1 and 5.1.1 to be specific.
I suppose we have a testing gap for upgrades. These are hard to do in an 
automated way, though. If someone has some cycles to think about how to do that 
in an IT that would be valuable and appreciated.
Or perhaps we do not have the right strategy in general and code and metadata 
are coupled too tightly...? Also something to think about.

Let's close the final Jiras soon. And the we can release a 
SNAPSHOT/beta/whatever for people to bang against.

And let's keep the test suite passing. And if you happen to look at some test 
code, also think about the performance. It is much more valuable to have the 
test suite that can pass in 1h than one that takes 2 or 3h to run.

One last question... Should we revert the ViewIndexId changes and push then to 
4.15.1/5.1.1? Is there a pressing need for those? (I suppose with the way we 
use Views at Salesforce there probably is, just want to confirm.)
-- Lars

On Friday, June 28, 2019, 2:50:06 PM PDT, Geoffrey Jacoby 
 wrote:  
 
 Lars,

That sounds good to me -- whether the thing people test is called "beta" or
"-SNAPSHOT", the important thing is that our code base is well-tested and,
as you say, something we all have confidence in.

In addition to splittable syscat (which I used as an example not because of
quality concerns but because it's both very large, very central and
one-way), other changes in 4.15 / 5.1 that might need attention for upgrade
testing are the optional increase of ViewIndexId from a short to an int
(PHOENIX-3547), and my own changes to fix a bug in ViewIndexId generation
in PHOENIX-5132 / 5138. (The bug fix was simple; making it upgrade
pre-existing view index sequences in a safe way was hard.) There are likely
others I've forgotten or don't know about.

The index changes also require upgrade and perf testing (some of which has
been done, with good results, but more to go), but the nice thing there is
that they're feature-flagged (opt-out for new tables/indexes, opt-in for
existing ones via the upgrade tool in PHOENIX-5333.) and operators can
switch back to the old design (even for new tables and indexes) if they
need to using the upgrade tool's rollback option. So once testing is
complete I think it's fine for them to go in 4.14.3.

Geoffrey




On Fri, Jun 28, 2019 at 12:20 PM la...@apache.org  wrote:

>  Any further comments?
> I offered to be the RM for 4.15.0, and I stand by that. I can't do it
> alone, though. Do we have consensus on the rough course of action below?
> Any other ideas? How far are we from a reasonable 4.15.0 release?
>
> IMHO we urgently need to apply some software engineering principles,
> namely an always releasable code base and small, frequent releases.
>
> -- Lars
>
>    On Thursday, June 27, 2019, 3:07:27 PM PDT, la...@apache.org <
> la...@apache.org> wrote:
>
>  Thanks Geoffrey.
> The damage is already done. We messed up and let it slide (multiple times,
> this is by no means the first time) and thus are in exactly the situation
> you outlined: No confidence in the code base.
> Now we can only look forward and get the code into a releasable state. The
> most important aspects are - as you point out, and I agree - getting
> confidence in splittable syscat and finishing the indexing work.
>
> In hindsight we should have done a release right before splittable syscat
> and perhaps one right after. Oh well. :)
>
> Could you mark the Jiras you remember with 4.15.0 and 5.1.0 fix versions
> (or are you saying you did already?)
> Since you say that we can release 4.14.3 with just the index changes, does
> that imply that you are mostly concerned about splittable syscat in 4.15.0
> and 5.1.0?
>
> I'm not a fan of a "beta" release, honestly. We can only do as good as we
> can and release a version that we believe in good conscience that there are
> no major issues. All releases will contain some bugs that are found later.
> It seems we are not even at that point yet... The good conscience part. :)
>
> How about we institute an immediate absolutely-no-new-feature policy for
> *all* of Phoenix until we have a releasable project? I'd be happy to
> enforce that. One cannot add new features to a code base that is not
> releasable/stable anyway. Until a few weeks ago we *never* had a passing
> test run. I really don't understand how we get here over and over again.
> But whatever, it's too late, and whining surely doesn't help.
>
> Lemme propose the following action plan then based on this and what you
> said:
> 1. We release 4.14.3 with just the index changes. Soon.
>
> 2. We immediately stop all new feature development in all branches
> (including 5.x, i.e. master)
>
> 3. We harden/test/etc splittable sy

Re: 4.15.0 and 5.1.0 releases

[la...@apache.org]

 Any further comments?
I offered to be the RM for 4.15.0, and I stand by that. I can't do it alone, 
though. Do we have consensus on the rough course of action below? Any other 
ideas? How far are we from a reasonable 4.15.0 release?

IMHO we urgently need to apply some software engineering principles, namely an 
always releasable code base and small, frequent releases.

-- Lars

On Thursday, June 27, 2019, 3:07:27 PM PDT, la...@apache.org 
 wrote:  
 
  Thanks Geoffrey.
The damage is already done. We messed up and let it slide (multiple times, this 
is by no means the first time) and thus are in exactly the situation you 
outlined: No confidence in the code base.
Now we can only look forward and get the code into a releasable state. The most 
important aspects are - as you point out, and I agree - getting confidence in 
splittable syscat and finishing the indexing work.

In hindsight we should have done a release right before splittable syscat and 
perhaps one right after. Oh well. :)

Could you mark the Jiras you remember with 4.15.0 and 5.1.0 fix versions (or 
are you saying you did already?)
Since you say that we can release 4.14.3 with just the index changes, does that 
imply that you are mostly concerned about splittable syscat in 4.15.0 and 5.1.0?

I'm not a fan of a "beta" release, honestly. We can only do as good as we can 
and release a version that we believe in good conscience that there are no 
major issues. All releases will contain some bugs that are found later.
It seems we are not even at that point yet... The good conscience part. :)

How about we institute an immediate absolutely-no-new-feature policy for *all* 
of Phoenix until we have a releasable project? I'd be happy to enforce that. 
One cannot add new features to a code base that is not releasable/stable 
anyway. Until a few weeks ago we *never* had a passing test run. I really don't 
understand how we get here over and over again. But whatever, it's too late, 
and whining surely doesn't help.

Lemme propose the following action plan then based on this and what you said:
1. We release 4.14.3 with just the index changes. Soon.

2. We immediately stop all new feature development in all branches (including 
5.x, i.e. master)

3. We harden/test/etc splittable syscat as well as other accumulated tech debt 
that we identify.

4. After we release 4.15.0 and 5.1.0 we allow feature work again.
5. Following those releases we do strictly monthly releases on all branches 
(and if we cannot do that, declare a branch dead)
Some of these (especially #5) might be radical, but we if we want to avoid this 
situation again we need to apply some rigor. As is Phoenix has been turning 
into an almost unmaintainable project over the past years, we need to actively 
counter that.

Cheers!

-- Lars

    On Thursday, June 27, 2019, 1:36:37 PM PDT, Geoffrey Jacoby 
 wrote:  
 
 Lars,

I agree 100% that we should have smaller, more frequent releases going
forward. As for this release, I have two concerns.

The first is indexes. I've added several JIRAs that had been incorrectly
not marked with a Fix Version to 4.15 / 5.1. These are all part of the
Self-Repairing Index project, which spans several JIRAs and whose first
major one (PHOENIX-5156, allowing newly created mutable indexes to
self-repair inconsistencies at read time) is already in 4.15 and 5.1.
Outstanding JIRAs include PHOENIX-5211 to extend the logic to immutable
indexes, and PHOENIX-5333, to give users a tool to convert their legacy
indexes to the new model. These are all under review and should land very
soon.

Especially given the multiple reports on the user list of operators
encountering index consistency problems (which I have also seen in my own
environments), I think it's important that our next release include these
fixes, and that they go out in a unified way.

The second concern is testing, particularly upgrade, perf and chaos
testing. In addition to the large index changes (for which I know some perf
work and live-cluster testing has been done, with more planned), there are
other major changes in 4.15 such as the splittable system catalog. If all
the issues on the current list were fixed, I'd still be reluctant to put
the bits into production without more due diligence. We've released
binaries with significant regressions in them that were missed in our test
suites before, and it's important to avoid that this time.

Yet Lars's point that we've waited far too long to release is of course
correct. Perhaps the solution is to do what the HBase community did when
the 2.x branch dragged out too long, and after the listed issues are Fixed,
we release an explicit beta, closed to new features, from which a final
release can graduate. In parallel, we could release a 4.14.3 with just the
index changes and the current diff from 4.14.2 so users get those faster.

Or maybe our testing's advanced further than I know about, and we're closer
to green than I think. Happy to hear everyone'

4.x-HBase-1.4 jenkins runs

[la...@apache.org]

Hi all,
in the past weeks we had mostly successful test runs on 4.x-HBase-1.4

Yet when you look at that build it's all read. The reason is that tests all run 
and finish and then Jenkins just hangs trying to archive the test artifacts for 
an hour or more. That only happens on the -1.4 branch (does not happen on -1.3, 
-1.5, or master branches).
Does anybody have an idea about this? Do we need to restrict the jenkinds VMs 
to run on? Or something else?

Thanks.
-- Lars

Re: 4.15.0 and 5.1.0 releases

[la...@apache.org]

 Thanks Geoffrey.
The damage is already done. We messed up and let it slide (multiple times, this 
is by no means the first time) and thus are in exactly the situation you 
outlined: No confidence in the code base.
Now we can only look forward and get the code into a releasable state. The most 
important aspects are - as you point out, and I agree - getting confidence in 
splittable syscat and finishing the indexing work.

In hindsight we should have done a release right before splittable syscat and 
perhaps one right after. Oh well. :)

Could you mark the Jiras you remember with 4.15.0 and 5.1.0 fix versions (or 
are you saying you did already?)
Since you say that we can release 4.14.3 with just the index changes, does that 
imply that you are mostly concerned about splittable syscat in 4.15.0 and 5.1.0?

I'm not a fan of a "beta" release, honestly. We can only do as good as we can 
and release a version that we believe in good conscience that there are no 
major issues. All releases will contain some bugs that are found later.
It seems we are not even at that point yet... The good conscience part. :)

How about we institute an immediate absolutely-no-new-feature policy for *all* 
of Phoenix until we have a releasable project? I'd be happy to enforce that. 
One cannot add new features to a code base that is not releasable/stable 
anyway. Until a few weeks ago we *never* had a passing test run. I really don't 
understand how we get here over and over again. But whatever, it's too late, 
and whining surely doesn't help.

Lemme propose the following action plan then based on this and what you said:
1. We release 4.14.3 with just the index changes. Soon.

2. We immediately stop all new feature development in all branches (including 
5.x, i.e. master)

3. We harden/test/etc splittable syscat as well as other accumulated tech debt 
that we identify.

4. After we release 4.15.0 and 5.1.0 we allow feature work again.
5. Following those releases we do strictly monthly releases on all branches 
(and if we cannot do that, declare a branch dead)
Some of these (especially #5) might be radical, but we if we want to avoid this 
situation again we need to apply some rigor. As is Phoenix has been turning 
into an almost unmaintainable project over the past years, we need to actively 
counter that.

Cheers!

-- Lars

On Thursday, June 27, 2019, 1:36:37 PM PDT, Geoffrey Jacoby 
 wrote:  
 
 Lars,

I agree 100% that we should have smaller, more frequent releases going
forward. As for this release, I have two concerns.

The first is indexes. I've added several JIRAs that had been incorrectly
not marked with a Fix Version to 4.15 / 5.1. These are all part of the
Self-Repairing Index project, which spans several JIRAs and whose first
major one (PHOENIX-5156, allowing newly created mutable indexes to
self-repair inconsistencies at read time) is already in 4.15 and 5.1.
Outstanding JIRAs include PHOENIX-5211 to extend the logic to immutable
indexes, and PHOENIX-5333, to give users a tool to convert their legacy
indexes to the new model. These are all under review and should land very
soon.

Especially given the multiple reports on the user list of operators
encountering index consistency problems (which I have also seen in my own
environments), I think it's important that our next release include these
fixes, and that they go out in a unified way.

The second concern is testing, particularly upgrade, perf and chaos
testing. In addition to the large index changes (for which I know some perf
work and live-cluster testing has been done, with more planned), there are
other major changes in 4.15 such as the splittable system catalog. If all
the issues on the current list were fixed, I'd still be reluctant to put
the bits into production without more due diligence. We've released
binaries with significant regressions in them that were missed in our test
suites before, and it's important to avoid that this time.

Yet Lars's point that we've waited far too long to release is of course
correct. Perhaps the solution is to do what the HBase community did when
the 2.x branch dragged out too long, and after the listed issues are Fixed,
we release an explicit beta, closed to new features, from which a final
release can graduate. In parallel, we could release a 4.14.3 with just the
index changes and the current diff from 4.14.2 so users get those faster.

Or maybe our testing's advanced further than I know about, and we're closer
to green than I think. Happy to hear everyone's thoughts.

Geoffrey

On Thu, Jun 27, 2019 at 10:26 AM la...@apache.org  wrote:

> Hi all,
> we're getting close. The test suite is passing fairly reliably now.(minus
> some strange failure to archive the artifact in -1.4 and PartialCommitIT
> failing in -1.3 only).
> I put a lot of effort into speeding up the tests and making them pass.
> Let's please (pretty please :) ) keep it that way.A passing, comprehensive
> test suite is key t

4.15.0 and 5.1.0 releases

[la...@apache.org]

Hi all,
we're getting close. The test suite is passing fairly reliably now.(minus some 
strange failure to archive the artifact in -1.4 and PartialCommitIT failing in 
-1.3 only).
I put a lot of effort into speeding up the tests and making them pass. Let's 
please (pretty please :) ) keep it that way.A passing, comprehensive test suite 
is key to frequent releases.

I also committed and push some issues to 4.15.1 and 5.1.1 already. But I can't 
do it alone.

There are 14 items to go for 4.15.0. Some of those are potentially 
serious.https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%204.15.0

And 26 items for 5.1.0
https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20fixVersion%20%3D%205.1.0

Let's make a final push and get these done (or moved to 4.15.1/5.1.1, resp)If 
you have any issues open, please either get them committed to move them to the 
next release.

And then let's try to never get into this situation again where we have a huge 
unreleased (and unreleasable) code base with 100's or 1000's of unreleased 
changes.
Thanks!
-- Lars

I'll be pushing some test speed-ups soon

[la...@apache.org]

Hi all,
I'll be pushing out out a set of changes to speed up various tests.Let's see 
how these go. If you see something strange, please let me know and I'll 
revert/fix.
Most test runs already finish in around 2h (instead of 3h), but there're still 
some bad offenders.

I'll do each test as a separate commit to they'll be easy to revert if we find 
something.
Thanks.

-- Lars

Re: Differences in 4.x branches

[la...@apache.org]

 
Let's fix PHOENIX-5228 at least doing what Andy suggested.(And of there were 
more changes we should have absolutely done what Andy suggested for all 4.x 
branches!)

Unfortunately it looks like neither -1.3, nor -1.4, or -1.5 are canonical. It's 
all over the place.
(And don't get me wrong, I'm thankful that someone picked up the menial of 
changing the logging! Let's just fix it and make it the same.) 

-- Lars

On Friday, June 14, 2019, 7:34:57 AM PDT, Geoffrey Jacoby 
 wrote:  
 
 Karan mentions on PHOENIX-4009 that it relies on libraries introduced in
HBase 1.4 and up and so is intentionally missing from 4.x-HBase-1.3. That
sort of divergence happens occasionally and is one of the reasons why we
have separate branches.

At a glance at PHOENIX-5228 though, I'd have expected that to be nearly
identical across versions.

Geoffrey

On Thu, Jun 13, 2019 at 10:41 PM la...@apache.org  wrote:

>  I think in the end it's mostly two jiras that cause the difference:
> PHOENIX-5228 - this one is weird since there seem to be so many
> unnecessary differences between the branches, and non seems to be
> canonically right. It's pretty big so I can perhaps see how this happened.
> PHOENIX-4009 - which was just not pushed to 1.3
> The rest are mostly minor changes that seem legit. If we can push
> PHOENIX-4009 to -1.3 as well, one could do a quick run between the branches
> and make PHOENIX-5228 the same, without the need to force-push.
>
> The difference in tests-failures are present, though. PartialCommitIT for
> example fails on the -1.3 branch only. And no green run of -1.4 or master,
> yet.
> I've done some work, too, to speed the test up. Some runs now finish in
> just over 2h now (never below 3h before).Oh got another green -1.5 run ;-)
>
> -- Lars
>
>    On Thursday, June 13, 2019, 8:51:00 PM PDT, Andrew Purtell <
> andrew.purt...@gmail.com> wrote:
>
>  What I would recommend is take on branch as the canonical branch, I guess
> the 1.5 one since Lars says it’s testing out green. Find the common
> ancestor prior to divergence. For all other branches reset the head to that
> ancestor and then pick forward from 1.5 one commit at a time, fixing up for
> compile issues due the the different HBase versions. Then make a push to
> resolve any lingering test issues. This means all but the 1.5 branch would
> have history rewritten and would be force pushed. However in exchange
> you’ll have common and consistent history on all.
>
> If this is an acceptable result I have undertaken this kind of janitorial
> work and would be pleased to offer my services for clean up duty. The ginsu
> I mean git knives for slicing and dicing history are sharp and at the
> ready. At the conclusion of the work you the Phoenix community will have
> only the last step to perform, which would be to resolve any lingering test
> issues.
>
>
> > On Jun 13, 2019, at 7:29 PM, "la...@apache.org" 
> wrote:
> >
> > Hi all,
> > we have three active 4.x branches: 4.x-HBase-1.3, 4.x-HBase-1.4,
> 4.x-HBase-1.5.
> > It looks like we're lacking _basic_ discipline here. There are patches
> in some branches but not in others, some patches are different between
> these branches for no good reason, different tests fail, etc.
> > I have run out of my available time to track these all down. I filed
> Jiras to that extend, but as is we're completely unable to release a
> coherent 4.15.0 version of Phoenix (it will be different between the three
> versions)
> >
> > We can either just declare 4.x-HBase-1.3 dead and align -1.4 and -1.5,
> or folks reading this can please push their changes everywhere, or ... I
> don't know we give up? Release anyway?
> > The 4.x-HBase-1.5 test suite is passing - I spent a lot of time on at
> least that. There are no excuses like "But it failed before!".(there's at
> least one known flapper, and I'll try to fix or disable that test)
> >
> > I'll stay to my threat and veto/revert any patch that changes that.If
> that ends up swimming against the stream and eats up too much of my time,
> I'll just give up!
> > -- Lars  

Re: Differences in 4.x branches

[la...@apache.org]

 I think in the end it's mostly two jiras that cause the difference:
PHOENIX-5228 - this one is weird since there seem to be so many unnecessary 
differences between the branches, and non seems to be canonically right. It's 
pretty big so I can perhaps see how this happened.
PHOENIX-4009 - which was just not pushed to 1.3
The rest are mostly minor changes that seem legit. If we can push PHOENIX-4009 
to -1.3 as well, one could do a quick run between the branches and make 
PHOENIX-5228 the same, without the need to force-push.

The difference in tests-failures are present, though. PartialCommitIT for 
example fails on the -1.3 branch only. And no green run of -1.4 or master, yet.
I've done some work, too, to speed the test up. Some runs now finish in just 
over 2h now (never below 3h before).Oh got another green -1.5 run ;-)

-- Lars

On Thursday, June 13, 2019, 8:51:00 PM PDT, Andrew Purtell 
 wrote:  
 
 What I would recommend is take on branch as the canonical branch, I guess the 
1.5 one since Lars says it’s testing out green. Find the common ancestor prior 
to divergence. For all other branches reset the head to that ancestor and then 
pick forward from 1.5 one commit at a time, fixing up for compile issues due 
the the different HBase versions. Then make a push to resolve any lingering 
test issues. This means all but the 1.5 branch would have history rewritten and 
would be force pushed. However in exchange you’ll have common and consistent 
history on all. 

If this is an acceptable result I have undertaken this kind of janitorial work 
and would be pleased to offer my services for clean up duty. The ginsu I mean 
git knives for slicing and dicing history are sharp and at the ready. At the 
conclusion of the work you the Phoenix community will have only the last step 
to perform, which would be to resolve any lingering test issues. 


> On Jun 13, 2019, at 7:29 PM, "la...@apache.org"  wrote:
> 
> Hi all,
> we have three active 4.x branches: 4.x-HBase-1.3, 4.x-HBase-1.4, 
> 4.x-HBase-1.5.
> It looks like we're lacking _basic_ discipline here. There are patches in 
> some branches but not in others, some patches are different between these 
> branches for no good reason, different tests fail, etc.
> I have run out of my available time to track these all down. I filed Jiras to 
> that extend, but as is we're completely unable to release a coherent 4.15.0 
> version of Phoenix (it will be different between the three versions)
> 
> We can either just declare 4.x-HBase-1.3 dead and align -1.4 and -1.5, or 
> folks reading this can please push their changes everywhere, or ... I don't 
> know we give up? Release anyway?
> The 4.x-HBase-1.5 test suite is passing - I spent a lot of time on at least 
> that. There are no excuses like "But it failed before!".(there's at least one 
> known flapper, and I'll try to fix or disable that test)
> 
> I'll stay to my threat and veto/revert any patch that changes that.If that 
> ends up swimming against the stream and eats up too much of my time, I'll 
> just give up!
> -- Lars  

Differences in 4.x branches

[la...@apache.org]

Hi all,
we have three active 4.x branches: 4.x-HBase-1.3, 4.x-HBase-1.4, 4.x-HBase-1.5.
It looks like we're lacking _basic_ discipline here. There are patches in some 
branches but not in others, some patches are different between these branches 
for no good reason, different tests fail, etc.
I have run out of my available time to track these all down. I filed Jiras to 
that extend, but as is we're completely unable to release a coherent 4.15.0 
version of Phoenix (it will be different between the three versions)

We can either just declare 4.x-HBase-1.3 dead and align -1.4 and -1.5, or folks 
reading this can please push their changes everywhere, or ... I don't know we 
give up? Release anyway?
The 4.x-HBase-1.5 test suite is passing - I spent a lot of time on at least 
that. There are no excuses like "But it failed before!".(there's at least one 
known flapper, and I'll try to fix or disable that test)

I'll stay to my threat and veto/revert any patch that changes that.If that ends 
up swimming against the stream and eats up too much of my time, I'll just give 
up!
-- Lars

Re: A successful jenkins test run, at last

[la...@apache.org]

 https://builds.apache.org/job/Phoenix-4.x-HBase-1.5/
The embedded image got lost.

On Monday, June 10, 2019, 10:12:49 AM PDT, William Shen 
 wrote:  
 
 Awesome news. Thanks Lars!
Not sure if I'm looking at the wrong place, but I don't think I found a
green build on jenkins yet. Is this change local only for now?
https://builds.apache.org/job/PreCommit-PHOENIX-Build/

On Mon, Jun 10, 2019 at 8:54 AM la...@apache.org  wrote:

> I finally managed to get a single successful test run. Yeah! :)
>
> The fact that this is worth a note to the @dev points to a larger problem,
> though... Our tests have been failing for months (or years? I don't
> remember that last successful run.)
>
> I'll continue to stabilize the tests (for 4.15.x and 5.1.x). I'll also
> keep an eye on the test runtimes.
>
> More importantly I will also start to veto and revert _any_ change that
> breaks a new test, other committers should please do the same. It is task
> of every contributor and committer to ensure that the test suite passes.
>
> Only that way can we hope to have more frequent releases.
>
> -- Lars
>
> And here's the proof:
>
> [image: Inline image]
>
>
>
  

A successful jenkins test run, at last

[la...@apache.org]

I finally managed to get a single successful test run. Yeah! :)

The fact that this is worth a note to the @dev points to a larger problem, 
though... Our tests have been failing for months (or years? I don't remember 
that last successful run.)

I'll continue to stabilize the tests (for 4.15.x and 5.1.x). I'll also keep an 
eye on the test runtimes.

More importantly I will also start to veto and revert _any_ change that breaks 
a new test, other committers should please do the same. It is task of every 
contributor and committer to ensure that the test suite passes.

Only that way can we hope to have more frequent releases.

-- Lars

And here's the proof:

Re: [VOTE] Release of Apache Phoenix 4.14.2 RC2

[la...@apache.org]

 +1 again based on inspection of the release artifacts.

On Thursday, May 23, 2019, 2:09:52 PM PDT, Thomas D'Silva 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.14.2 RC2. This is a patch
release of Phoenix 4.14 and is compatible with Apache HBase 1.3 and 1.4.
The release includes both a source-only release and a convenience binary
release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
several critical bug fixes.

The source tarball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc2/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc2/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc2/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc2/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12344379

Artifacts are signed with my "CODE SIGNING KEY": DFD86C02

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=13ce47cf4214c9c78e8fec5f96fb861410677817
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=a86206a006c85afd5884a015d6fe452d823ac626

Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team
  

Re: [VOTE] Release of Apache Phoenix 4.14.2 RC1

[la...@apache.org]

 I just inspected the artifacts. Based on that: +1

On Saturday, May 18, 2019, 12:38:47 AM PDT, Thomas D'Silva 
 wrote:  
 
 Hello Everyone,

This is a call for a vote on Apache Phoenix 4.14.2 RC1. This is a patch
release of Phoenix 4.14 and is compatible with Apache HBase 1.3 and 1.4.
The release includes both a source-only release and a convenience binary
release for each supported HBase version.

This release has feature parity with supported HBase versions and includes
several critical bug fixes.

The source tarball, including signatures, digests, etc can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc1/src/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc1/src/

The binary artifacts can be found at:
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.3-rc1/bin/
https://dist.apache.org/repos/dist/dev/phoenix/apache-phoenix-4.14.2-HBase-1.4-rc1/bin/

For a complete list of changes, see:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315120=12344379

Artifacts are signed with my "CODE SIGNING KEY": DFD86C02

KEYS file available here:
https://dist.apache.org/repos/dist/dev/phoenix/KEYS

The hash and tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=e3505d91e46de1a1756a145d396f27a3c70e927f
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=4b717825fb284c1f9fbdeee3d0e5391f5baf13bb


Vote will be open for at least 72 hours. Please vote:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
The Apache Phoenix Team
  

Re: Handling of HBase version specifics.

[la...@apache.org]

 Yes and yes :)
I think we should make a sweep and fix (or disable) all failing tests.Then 
absolutely let no further commit through that fails any test.
Our Jenkins builds do seem to have a lot of environmental issues, too, though.

As for the supported HBase versions. There's a line of course.I think we're 
down to HBase 2.x (master), HBase 1.5.x, HBase 1.4.x, and HBase 1.3.xHopefully 
Phoenix has no direct dependency on what specific version of HDFS that actual 
HBase instance is using.

Even then it's four branches to maintain separately. We have the test load 
right now anyway, it's just spread over 4 branches.

-- Lars

On Sunday, May 19, 2019, 9:18:19 PM PDT, James Taylor 
 wrote:  
 
 I think it’s a good idea, but even before this you have to ask when a
Jenkins build actually succeeded. How about solving that first?

On Sun, May 19, 2019 at 1:34 PM Nick Dimiduk  wrote:

> I think this is generally a good idea for managing multiple target
> runtimes.
>
> One question I have though: is it really necessary that we support so many
> release branches and so many compile targets? What about the versions of
> Hadoop underneath each of those versions of HBase? Are we committed to
> running against ever version of HDFS that each of those HBase releases
> support? The test load will be massive and the suite itself must be
> rock-solid in order to run across so many permutations. I fear that’s an
> unreasonable burden for an open source community.
>
> Thanks,
> Nick
>
> On Sat, May 18, 2019 at 11:36 AM Thomas D'Silva
>  wrote:
>
> > +1, I think this is a good idea. This would make it easier to contribute
> > and commit since you would only have to create a single patch.
> > The tests would take longer to run (1.3, 1.4, 1.5 and 2.x). We should
> make
> > sure our precommit build will run the tests for all the modules.
> >
> >
> >
> > On Fri, May 17, 2019 at 11:23 AM la...@apache.org 
> > wrote:
> >
> > > Hi all,
> > > historically we have a branch of each version of HBase we want to
> > > support.As a result we have many branches, committing is a hassle and
> it
> > is
> > > easy to miss a change across branches.
> > > Instead we could have a maven module per version of HBase we want to
> > > support and move the version dependent code there.Take a look at what
> > > Tephra is doing: https://github.com/apache/incubator-tephra
> > > They have a compat module for each supported version of HBase, and
> > version
> > > dependent code is "simply" copied into those modules.There's still
> > > duplicate code, but at least there's one branch to maintain.
> > > It's somewhat of a bigger project now.
> > > Thoughts?
> > > -- Lars
> > >
> >
>  

Handling of HBase version specifics.

[la...@apache.org]

Hi all,
historically we have a branch of each version of HBase we want to support.As a 
result we have many branches, committing is a hassle and it is easy to miss a 
change across branches.
Instead we could have a maven module per version of HBase we want to support 
and move the version dependent code there.Take a look at what Tephra is doing: 
https://github.com/apache/incubator-tephra
They have a compat module for each supported version of HBase, and version 
dependent code is "simply" copied into those modules.There's still duplicate 
code, but at least there's one branch to maintain.
It's somewhat of a bigger project now.
Thoughts?
-- Lars

Re: [DISCUSS] Drop support for HBase-1.2

[la...@apache.org]

 +1
On Monday, May 13, 2019, 1:31:35 PM EDT, Vincent Poon 
 wrote:  
 
 +1

On Sun, May 12, 2019 at 8:18 PM Andrew Purtell 
wrote:

> +1
>
> I would think it would be a drain on committer time to keep having to
> accommodate interface differences on the EOL line.
>
> > On May 10, 2019, at 1:28 PM, Thomas D'Silva 
> wrote:
> >
> > Since HBase 1.2 is now end of life and we are creating a new branch to
> > support HBase-1.5(PHOENIX-5277), I think we should drop the HBase-1.2
> > branches. What do folks think?
> >
> > Thanks,
> > Thomas
>
  

Re: Phoenix developer Meetup

[la...@apache.org]

 Hi all,
here's the link to the video conference meet.google.com/nxw-kcrq-skd.The video 
will also be recorded.

If you can, come in person, though, face-to-face discussions are usually 
better, and there'll be food :)
-- Lars
On Wednesday, November 14, 2018, 8:45:43 AM PST, Bin Shi 
 wrote:  
 
 Same question -- is it possible for people who is in different city to join
remotely?

Thanks,

*Bin Shi **LinkedIn <https://www.linkedin.com/in/bin-shi-63a118108/>*
PMTS | Infrastructure Engineering
*Salesforce (Seattle|Bellevue)*
*Cell:* 425-247-4348
*Email: *b...@salesforce.com


On Tue, Nov 13, 2018 at 11:50 PM Jaanai Zhang 
wrote:

> How to connect video conference？
>
> 
>    Jaanai Zhang
>    Best regards!
>
>
>
> la...@apache.org  于2018年11月10日周六 上午6:48写道：
>
> >  To confirm:
> > Phoenix DEV Meetup.
> > Wednesday, November 14th, 4pm, Salesforce Tower (415 Mission St.) (17th
> > floor, sorry not the top floor). We'll meet in the lobby to get visitor
> > badges.Please try to be on time, or let me know when you'll be later
> > (someone will have to come down to let you up)
> > Thanks.
> > -- Lars
> >    On Friday, November 9, 2018, 10:30:09 AM PST, la...@apache.org <
> > la...@apache.org> wrote:
> >
> >  Great.
> > And sorry for the folks in other TZs, it's impossible to find a time for
> > everyone.I'll try to record the session via hangouts and post the
> recording
> > here, and I will post the notes.
> > Here's an initial agenda, distilled and grouped from all your suggested
> > topics:(I plan to keep it free-form, so we can get some good discussions,
> > but if you want prepare _short_ slides/docs or proposals for the topics
> you
> > care about that is of course very welcome!)
> >
> > Intro
> >
> >  - Open Source at Salesforce (Chris Kelly)
> >  - Phoenix usage at Salesforce (Lars Hofhansl)
> >
> > Round-table
> >
> >  - Introductions
> >  - Current thoughts and pain points
> >
> > Deep Dives
> >
> > (topics derived from requests from the community)
> >
> > Direction and Future
> >
> >  - Status of 4.x and master branches
> >  - Current challenges (public cloud?, performance?, community?, SQL
> > coverage?, scale?)
> >  - Secondary Indexing (Lars Hofhansl)
> >  - Future direction. Where do we want Phoenix to be in 1 years, 2 years,
> > 5 years?
> >  - Feature gaps in multi-tenancy. Fat client vs thin
> >  - Feature resiliency / hardening (general)
> >  - integration with Data Analytics tool like spark.
> >
> > Project Management
> >
> >  - Release cadence (need RMs that spend time on this)
> >  - Improve mailing list support
> >  - PR review rotation
> >  - Nominate new committers
> >  - Branch cleanup
> >  - checkin criteria and testing (IT vs. unit tests)
> >
> > Project Cleanliness
> >
> >  - Fix test flappers and overall improvements overall test infra
> >  - Refactoring, correctness, code extensibility
> >  - Potential refactoring work
> >  - Perf tool for Phoenix Query Server
> >  - Thoughts on test infra
> >
> >
> >
> >
> >
> >    On Wednesday, November 7, 2018, 3:45:44 PM PST, Josh Elser <
> > els...@apache.org> wrote:
> >
> >  My schedule may be getting switched around. Might be able to make it
> > physically!
> >
> > Marked myself as a "M" for maybe on the spreadsheet :)
> >
> > On 11/6/18 7:07 PM, la...@apache.org wrote:
> > >  Ok. The room is booked for Wednesday November 14th from 4-7pm, in the
> > Salesforce Tower (but not the top floor :( )
> > > If you can confirm your attendance here (see the extra column on the
> > right):
> >
> https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit#gid=0
> > >
> > > That would be great for planning food, etc. I will send out an agenda.
> > > Thanks.
> > > -- Lars
> > >      On Tuesday, October 23, 2018, 4:01:33 PM PDT, la...@apache.org <
> > la...@apache.org> wrote:
> > >
> > >    This is still on.
> > > I'm trying to get us a spot in the Salesforce tower top floor. As you
> > can imagine it's a coveted spot.If I can't get a time slot there in the
> > next week I'll book a "usual" conference room.
> > > Stay tuned.
> > > Thanks.
> > > -- Lars
> > >
> > >
> > >      On Friday, October 5, 2018, 1:01:06 PM PDT, la...@apache.org <
> > la...@apache.org

Re: Phoenix developer Meetup

[la...@apache.org]

 To confirm:
Phoenix DEV Meetup.
Wednesday, November 14th, 4pm, Salesforce Tower (415 Mission St.) (17th floor, 
sorry not the top floor). We'll meet in the lobby to get visitor badges.Please 
try to be on time, or let me know when you'll be later (someone will have to 
come down to let you up)
Thanks.
-- Lars
On Friday, November 9, 2018, 10:30:09 AM PST, la...@apache.org 
 wrote:  
 
  Great.
And sorry for the folks in other TZs, it's impossible to find a time for 
everyone.I'll try to record the session via hangouts and post the recording 
here, and I will post the notes.
Here's an initial agenda, distilled and grouped from all your suggested 
topics:(I plan to keep it free-form, so we can get some good discussions, but 
if you want prepare _short_ slides/docs or proposals for the topics you care 
about that is of course very welcome!)

Intro
  
  - Open Source at Salesforce (Chris Kelly)
  - Phoenix usage at Salesforce (Lars Hofhansl)

Round-table
  
  - Introductions
  - Current thoughts and pain points

Deep Dives

(topics derived from requests from the community)

Direction and Future
  
  - Status of 4.x and master branches
  - Current challenges (public cloud?, performance?, community?, SQL coverage?, 
scale?)
  - Secondary Indexing (Lars Hofhansl)
  - Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 
years? 
  - Feature gaps in multi-tenancy. Fat client vs thin
  - Feature resiliency / hardening (general)
  - integration with Data Analytics tool like spark.

Project Management
  
  - Release cadence (need RMs that spend time on this)
  - Improve mailing list support
  - PR review rotation
  - Nominate new committers
  - Branch cleanup
  - checkin criteria and testing (IT vs. unit tests)

Project Cleanliness
  
  - Fix test flappers and overall improvements overall test infra
  - Refactoring, correctness, code extensibility
  - Potential refactoring work
  - Perf tool for Phoenix Query Server
  - Thoughts on test infra





    On Wednesday, November 7, 2018, 3:45:44 PM PST, Josh Elser 
 wrote:  
 
 My schedule may be getting switched around. Might be able to make it 
physically!

Marked myself as a "M" for maybe on the spreadsheet :)

On 11/6/18 7:07 PM, la...@apache.org wrote:
>  Ok. The room is booked for Wednesday November 14th from 4-7pm, in the 
>Salesforce Tower (but not the top floor :( )
> If you can confirm your attendance here (see the extra column on the 
> right):https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit#gid=0
> 
> That would be great for planning food, etc. I will send out an agenda.
> Thanks.
> -- Lars
>      On Tuesday, October 23, 2018, 4:01:33 PM PDT, la...@apache.org 
> wrote:
>  
>    This is still on.
> I'm trying to get us a spot in the Salesforce tower top floor. As you can 
> imagine it's a coveted spot.If I can't get a time slot there in the next week 
> I'll book a "usual" conference room.
> Stay tuned.
> Thanks.
> -- Lars
> 
> 
>      On Friday, October 5, 2018, 1:01:06 PM PDT, la...@apache.org 
> wrote:
>  
>    Last call :)
>      On Monday, October 1, 2018, 10:15:07 AM PDT, la...@apache.org 
> wrote:
>  
>    10 people signed up so far.This is a good chance to make your voice heard, 
>give input, and help point the project in the right direction going forward.
> -- Lars
> 
>      On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
> wrote:
>  
>  Hi all,
> I'm planning to put together a Phoenix developer meetup at the Salesforce 
> office (with video conference for those who cannot attend in person) in the 
> next few weeks.
> If you're interested please put your name in this 
> spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing
> 
> This is a chance to get all those who contribute to Phoenix together. There 
> will also be food. :)I will leave the spreadsheet up for one week - until 
> Friday October 5th.
> Possible agenda:- Round-table- Status of 4.x and master branches- Current 
> challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
> Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 
> years?- more...
> Thanks.
> -- Lars
>        
> 
    

Re: [VOTE] Release of Apache Phoenix 4.14.1 RC0

[la...@apache.org]

 Belated +1
Ran my usually tests. With upserts, deleted, global and local indexes, larger 
upsert/selects, etc. Nothing undue in the logs, all works.
-- Lars

On Thursday, November 8, 2018, 3:55:07 PM PST, Vincent Poon 
 wrote:  
 
 The vote is now closed and passes with 4 binding +1s and no 0 or -1 votes.
Thanks to those who voted. We'll work on pushing out the bits next.


On Thu, Nov 8, 2018 at 3:52 PM Vincent Poon 
wrote:

> +1
> Ran mvn verify on all branches. A few errors passed after manually
> running.  Otherwise everything passed.
>
> On Wed, Nov 7, 2018 at 10:12 AM James Taylor 
> wrote:
>
>> +1
>>
>> Ran various manual sanity tests around loading/querying/indexing data
>>
>> On Tue, Nov 6, 2018 at 3:53 PM Thomas D'Silva 
>> wrote:
>>
>> > +1
>> >
>> > Ran some manual tests with indexes, sequences.
>> > Regarding Artem's question in PhoenixDatabaseMetdata.getDriverVersion()
>> we
>> > only return the majorVersion.minorVersion
>> > perhaps you can file a JIRA to also inlcude the patch version.
>> >
>> > On Fri, Nov 2, 2018 at 11:47 AM, Josh Elser  wrote:
>> >
>> > > +1 (binding)
>> > >
>> > > Some observations:
>> > > * Some cruft (0.98, 1.2, and 1.3 seem to have this), not a blocker,
>> > > but good to ensure is cleaned up in the future:
>> > > ./bin/phoenix_utils.pyc
>> > >
>> > > Everything else looks good to me.
>> > >
>> > > - Josh
>> > >
>> > > On Tue, Oct 30, 2018 at 4:47 PM Vincent Poon <
>> vincent.poon...@gmail.com>
>> > > wrote:
>> > > >
>> > > > Hello Everyone,
>> > > >
>> > > > This is a call for a vote on Apache Phoenix 4.14.1 RC0. This is a
>> patch
>> > > > release of Phoenix 4.14 and is compatible with Apache HBase 0.98,
>> 1.1,
>> > > 1.2,
>> > > > 1.3, 1.4. The release includes both a
>> > > > source-only release and a convenience binary release for each
>> supported
>> > > > HBase version.
>> > > >
>> > > > This release has feature parity with supported HBase versions and
>> > > includes
>> > > > critical bug fixes for secondary indexes.
>> > > >
>> > > > The source tarball, including signatures, digests, etc can be found
>> at:
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-0.98-rc0/src/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.1-rc0/src/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.2-rc0/src/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.3-rc0/src/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.4-rc0/src/
>> > > >
>> > > > The binary artifacts can be found at:
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-0.98-rc0/bin/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.1-rc0/bin/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.2-rc0/bin/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.3-rc0/bin/
>> > > > https://dist.apache.org/repos/dist/dev/phoenix/apache-
>> > > phoenix-4.14.1-HBase-1.4-rc0/bin/
>> > > >
>> > > > For a complete list of changes, see:
>> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>> > > projectId=12315120=12343452
>> > > >
>> > > > Release artifacts are signed with the following key:
>> > > > https://people.apache.org/keys/committer/vincentpoon.asc
>> > > > https://dist.apache.org/repos/dist/release/phoenix/KEYS
>> > > >
>> > > > The hash and tag to be voted upon:
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=
>> > > b27613ea409f81cd2d5a51e4ab14b1ca4d81f478
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;
>> > > h=refs/tags/v4.14.1-HBase-0.98-rc0
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=
>> > > ca3597d842d00fbb4cda5adb84fcb792ab06f3a2
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;
>> > > h=refs/tags/v4.14.1-HBase-1.1-rc0
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=
>> > > 537ad56e0683fedf252aef49dd82ca2851151385
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;
>> > > h=refs/tags/v4.14.1-HBase-1.2-rc0
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=
>> > > 1c06ebc1c54a736b3c390c58ca949450bf16e6a7
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;
>> > > h=refs/tags/v4.14.1-HBase-1.3-rc0
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=
>> > > c5ca547923bedf28a2d1b96362adb7e5e40dbd0c
>> > > > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=tag;
>> > > h=refs/tags/v4.14.1-HBase-1.4-rc0
>> > > >
>> > > >
>> > > > Vote will be open for at least 72 hours. Please vote:
>> > > >
>> > > > [ ] +1 approve
>> > > > [ ] +0 no opinion
>> > > > [ ] -1 disapprove (and reason why)
>> > > >
>> > > > Thanks,

Re: Phoenix developer Meetup

[la...@apache.org]

 Great.
And sorry for the folks in other TZs, it's impossible to find a time for 
everyone.I'll try to record the session via hangouts and post the recording 
here, and I will post the notes.
Here's an initial agenda, distilled and grouped from all your suggested 
topics:(I plan to keep it free-form, so we can get some good discussions, but 
if you want prepare _short_ slides/docs or proposals for the topics you care 
about that is of course very welcome!)

Intro
   
   - Open Source at Salesforce (Chris Kelly)
   - Phoenix usage at Salesforce (Lars Hofhansl)

Round-table
   
   - Introductions
   - Current thoughts and pain points

Deep Dives

(topics derived from requests from the community)

Direction and Future
   
   - Status of 4.x and master branches
   - Current challenges (public cloud?, performance?, community?, SQL 
coverage?, scale?)
   - Secondary Indexing (Lars Hofhansl)
   - Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 
years? 
   - Feature gaps in multi-tenancy. Fat client vs thin
   - Feature resiliency / hardening (general)
   - integration with Data Analytics tool like spark.

Project Management
   
   - Release cadence (need RMs that spend time on this)
   - Improve mailing list support
   - PR review rotation
   - Nominate new committers
   - Branch cleanup
   - checkin criteria and testing (IT vs. unit tests)

Project Cleanliness
   
   - Fix test flappers and overall improvements overall test infra
   - Refactoring, correctness, code extensibility
   - Potential refactoring work
   - Perf tool for Phoenix Query Server
   - Thoughts on test infra





On Wednesday, November 7, 2018, 3:45:44 PM PST, Josh Elser 
 wrote:  
 
 My schedule may be getting switched around. Might be able to make it 
physically!

Marked myself as a "M" for maybe on the spreadsheet :)

On 11/6/18 7:07 PM, la...@apache.org wrote:
>  Ok. The room is booked for Wednesday November 14th from 4-7pm, in the 
>Salesforce Tower (but not the top floor :( )
> If you can confirm your attendance here (see the extra column on the 
> right):https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit#gid=0
> 
> That would be great for planning food, etc. I will send out an agenda.
> Thanks.
> -- Lars
>      On Tuesday, October 23, 2018, 4:01:33 PM PDT, la...@apache.org 
> wrote:
>  
>    This is still on.
> I'm trying to get us a spot in the Salesforce tower top floor. As you can 
> imagine it's a coveted spot.If I can't get a time slot there in the next week 
> I'll book a "usual" conference room.
> Stay tuned.
> Thanks.
> -- Lars
> 
> 
>      On Friday, October 5, 2018, 1:01:06 PM PDT, la...@apache.org 
> wrote:
>  
>    Last call :)
>      On Monday, October 1, 2018, 10:15:07 AM PDT, la...@apache.org 
> wrote:
>  
>    10 people signed up so far.This is a good chance to make your voice heard, 
>give input, and help point the project in the right direction going forward.
> -- Lars
> 
>      On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
> wrote:
>  
>  Hi all,
> I'm planning to put together a Phoenix developer meetup at the Salesforce 
> office (with video conference for those who cannot attend in person) in the 
> next few weeks.
> If you're interested please put your name in this 
> spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing
> 
> This is a chance to get all those who contribute to Phoenix together. There 
> will also be food. :)I will leave the spreadsheet up for one week - until 
> Friday October 5th.
> Possible agenda:- Round-table- Status of 4.x and master branches- Current 
> challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
> Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 
> years?- more...
> Thanks.
> -- Lars
>        
> 
  

Re: Phoenix developer Meetup

[la...@apache.org]

 Ok. The room is booked for Wednesday November 14th from 4-7pm, in the 
Salesforce Tower (but not the top floor :( )
If you can confirm your attendance here (see the extra column on the 
right):https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit#gid=0

That would be great for planning food, etc. I will send out an agenda.
Thanks.
-- Lars
On Tuesday, October 23, 2018, 4:01:33 PM PDT, la...@apache.org 
 wrote:  
 
  This is still on.
I'm trying to get us a spot in the Salesforce tower top floor. As you can 
imagine it's a coveted spot.If I can't get a time slot there in the next week 
I'll book a "usual" conference room.
Stay tuned.
Thanks.
-- Lars


    On Friday, October 5, 2018, 1:01:06 PM PDT, la...@apache.org 
 wrote:  
 
  Last call :)
    On Monday, October 1, 2018, 10:15:07 AM PDT, la...@apache.org 
 wrote:  
 
  10 people signed up so far.This is a good chance to make your voice heard, 
give input, and help point the project in the right direction going forward.
-- Lars

    On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
 wrote:  
 
 Hi all,
I'm planning to put together a Phoenix developer meetup at the Salesforce 
office (with video conference for those who cannot attend in person) in the 
next few weeks.
If you're interested please put your name in this 
spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing

This is a chance to get all those who contribute to Phoenix together. There 
will also be food. :)I will leave the spreadsheet up for one week - until 
Friday October 5th.
Possible agenda:- Round-table- Status of 4.x and master branches- Current 
challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 years?- 
more...
Thanks.
-- Lars
      

Re: Phoenix developer Meetup

[la...@apache.org]

 This is still on.
I'm trying to get us a spot in the Salesforce tower top floor. As you can 
imagine it's a coveted spot.If I can't get a time slot there in the next week 
I'll book a "usual" conference room.
Stay tuned.
Thanks.
-- Lars


On Friday, October 5, 2018, 1:01:06 PM PDT, la...@apache.org 
 wrote:  
 
  Last call :)
    On Monday, October 1, 2018, 10:15:07 AM PDT, la...@apache.org 
 wrote:  
 
  10 people signed up so far.This is a good chance to make your voice heard, 
give input, and help point the project in the right direction going forward.
-- Lars

    On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
 wrote:  
 
 Hi all,
I'm planning to put together a Phoenix developer meetup at the Salesforce 
office (with video conference for those who cannot attend in person) in the 
next few weeks.
If you're interested please put your name in this 
spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing

This is a chance to get all those who contribute to Phoenix together. There 
will also be food. :)I will leave the spreadsheet up for one week - until 
Friday October 5th.
Possible agenda:- Round-table- Status of 4.x and master branches- Current 
challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 years?- 
more...
Thanks.
-- Lars
     

Re: Phoenix developer Meetup

[la...@apache.org]

 Last call :)
On Monday, October 1, 2018, 10:15:07 AM PDT, la...@apache.org 
 wrote:  
 
  10 people signed up so far.This is a good chance to make your voice heard, 
give input, and help point the project in the right direction going forward.
-- Lars

    On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
 wrote:  
 
 Hi all,
I'm planning to put together a Phoenix developer meetup at the Salesforce 
office (with video conference for those who cannot attend in person) in the 
next few weeks.
If you're interested please put your name in this 
spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing

This is a chance to get all those who contribute to Phoenix together. There 
will also be food. :)I will leave the spreadsheet up for one week - until 
Friday October 5th.
Possible agenda:- Round-table- Status of 4.x and master branches- Current 
challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 years?- 
more...
Thanks.
-- Lars
    

Re: Phoenix developer Meetup

[la...@apache.org]

 10 people signed up so far.This is a good chance to make your voice heard, 
give input, and help point the project in the right direction going forward.
-- Lars

On Friday, September 28, 2018, 10:39:41 AM PDT, la...@apache.org 
 wrote:  
 
 Hi all,
I'm planning to put together a Phoenix developer meetup at the Salesforce 
office (with video conference for those who cannot attend in person) in the 
next few weeks.
If you're interested please put your name in this 
spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing

This is a chance to get all those who contribute to Phoenix together. There 
will also be food. :)I will leave the spreadsheet up for one week - until 
Friday October 5th.
Possible agenda:- Round-table- Status of 4.x and master branches- Current 
challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 years?- 
more...
Thanks.
-- Lars
  

Phoenix developer Meetup

[la...@apache.org]

Hi all,
I'm planning to put together a Phoenix developer meetup at the Salesforce 
office (with video conference for those who cannot attend in person) in the 
next few weeks.
If you're interested please put your name in this 
spreadsheet:https://docs.google.com/spreadsheets/d/1j4QSk53B0ZIl_qq1XX3oB2f-Tyqp93ilJYZKG84_Amg/edit?usp=sharing

This is a chance to get all those who contribute to Phoenix together. There 
will also be food. :)I will leave the spreadsheet up for one week - until 
Friday October 5th.
Possible agenda:- Round-table- Status of 4.x and master branches- Current 
challenges (public cloud?, performance?, community?, SQL coverage?, scale?)- 
Future direction. Where do we want Phoenix to be in 1 years, 2 years, 5 years?- 
more...
Thanks.
-- Lars