Re: PQS + Kerberos problems

2019-05-28 Thread Aleksandr Saraseka 
Thank you Josh, that helps a lot.
We have Query Server on a dedicated server and none of existing guides have
an information that we need to have core-site.xml
with hadoop.security.authentication option set to kerberos.

On Tue, May 28, 2019 at 11:59 PM Josh Elser  wrote:

> Make sure you have authorization set up correctly between PQS and HBase.
>
> Specifically, you must have the appropriate Hadoop proxyuser rules set
> up in core-site.xml so that HBase will allow PQS to impersonate the PQS
> end-user.
>
> On 5/14/19 11:04 AM, Aleksandr Saraseka wrote:
> > Hello, I have HBase + PQS 4.14.1
> > If I'm trying to connect by think client - everything works, but if I'm
> > using thin client in PQS logs I can see continuous INFO messages
> > 2019-05-14 13:53:58,701 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=10, retries=35, started=48292 ms ago, cancelled=false, msg=
> > ...
> > 2019-05-14 14:18:41,446 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=33, retries=35, started=510325 ms ago, cancelled=false, msg=
> > 2019-05-14 14:19:01,489 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=34, retries=35, started=530368 ms ago, cancelled=false, msg=
> > ...
> > 2019-05-14 14:18:41,446 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=33, retries=35, started=510325 ms ago, cancelled=false, msg=
> > 2019-05-14 14:19:01,489 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=34, retries=35, started=530368 ms ago, cancelled=false, msg=
> > 2019-05-14 14:19:50,139 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=10, retries=35, started=48480 ms ago, cancelled=false, msg=row
> > 'SYSTEM:CATALOG,,' on table 'hbase:meta' at
> > region=hbase:meta,,1.1588230740, hostname=datanode-001.fqdn.com
> > ,60020,1557323271824, seqNum=0
> > 2019-05-14 14:20:10,333 INFO
> > org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception,
> > tries=11, retries=35, started=68676 ms ago, cancelled=false, msg=row
> > 'SYSTEM:CATALOG,,' on table 'hbase:meta' at
> > region=hbase:meta,,1.1588230740, hostname=datanode-001.fqdn.com
> > ,60020,1557323271824, seqNum=0
> >
> > *Hbase security logs:*
> > 2019-05-14 14:42:19,524 INFO
> > SecurityLogger.org.apache.hadoop.hbase.Server: Auth successful for
> > HTTP/phoenix-queryserver-fqdn@realm.com
> >  (auth:KERBEROS)
> > 2019-05-14 14:42:19,524 INFO
> > SecurityLogger.org.apache.hadoop.hbase.Server: Connection from
> > 10.252.16.253 port: 41040 with version info: version: "1.2.0-cdh5.14.2"
> > url:
> >
> "file:///data/jenkins/workspace/generic-binary-tarball-and-maven-deploy/CDH5.14.2-Packaging-HBase-2018-03-27_13-15-05/hbase-1.2.0-cdh5.14.2"
>
> > revision: "Unknown" user: "jenkins" date: "Tue Mar 27 13:31:54 PDT 2018"
> > src_checksum: "05e6e90e06dd7796f56067208a9bf2aa"
> > 2019-05-14 14:42:29,634 INFO
> > SecurityLogger.org.apache.hadoop.hbase.Server: Auth successful for
> > HTTP/phoenix-queryserver-fqdn@realm.com
> >  (auth:KERBEROS)
> > 2019-05-14 14:42:29,635 INFO
> > SecurityLogger.org.apache.hadoop.hbase.Server: Connection from
> > 10.252.16.253 port: 41046 with version info: version: "1.2.0-cdh5.14.2"
> > url:
> >
> "file:///data/jenkins/workspace/generic-binary-tarball-and-maven-deploy/CDH5.14.2-Packaging-HBase-2018-03-27_13-15-05/hbase-1.2.0-cdh5.14.2"
>
> > revision: "Unknown" user: "jenkins" date: "Tue Mar 27 13:31:54 PDT 2018"
> > src_checksum: "05e6e90e06dd7796f56067208a9bf2aa"
> >
> >
> > *thin client logs:*
> > 19/05/14 14:10:08 DEBUG execchain.MainClientExec: Proxy auth state:
> > UNCHALLENGED
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> POST / HTTP/1.1
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Content-Length:
> 137
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Content-Type:
> > application/octet-stream
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Host:
> > host-fqdn.com:8765 
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Connection:
> > Keep-Alive
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> User-Agent:
> > Apache-HttpClient/4.5.2 (Java/1.8.0_161)
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >>
> > Accept-Encoding: gzip,deflate
> > 19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Authorization:
> > Negotiate
> >
> YIICtQYGKwYBBQUCoIICqTCCAqWgDTALBgkqhkiG9xIBAgKhBAMCAXaiggKMBIICiGCCAoQGCSqGSIb3EgECAgEAboICczCCAm+gAwIBBaEDAgEOogcDBQAgo4IBgGGCAXwwggF4oAMCAQWhEBsOREFUQVNZUy5DRi5XVEaiQDA+oAMCAQChNzA1GwRIVFRQGy1kYXRhc3lzLXNlY3VyZS1odWUwMDEtc3RnLmMuY2Ytc3RhZ2UuaW50ZXJuYWyjggEbMIIBF6ADAgESoQMCAQOiggEJBIIBBTmyywjx8OcQBfIt2AccWAVPKmyf/UM5lDgRUs9cUixE35x6wl9EoIX6XTsw2hO4ES

[ANNOUNCE] Apache Phoenix 4.14.2 released

2019-05-28 Thread Thomas D'Silva 
The Apache Phoenix team is pleased to announce the immediate availability
of the 4.14.2 patch release. Apache Phoenix enables SQL-based OLTP and
operational analytics for Apache Hadoop using Apache HBase as its backing
store and providing integration with other projects in the Apache ecosystem
such as Spark, Hive, Pig, Flume, and MapReduce.

This patch release has feature parity with supported HBase versions and
includes critical bug fixes for secondary indexes.

Download source and binaries here [1].

Thanks,
Thomas (on behalf of the Apache Phoenix team)

[1] http://phoenix.apache.org/download.html

Re: Fwd: DELIVERY FAILURE: Error transferring to QCMBSJ601.HERMES.SI.SOCGEN; Maximum hop count exceeded. Message probably in a routing loop.

2019-05-28 Thread Josh Elser 
https://issues.apache.org/jira/browse/INFRA-18170 still pending. Need to 
poke.


On 5/23/19 2:49 PM, William Shen wrote:

Josh,

Any luck on getting Infra to help, or finding a moderator? I'm still 
getting these spam... (Who are the moderators anyway?)


Thanks,

- Will

On Fri, Apr 5, 2019 at 3:38 PM William Shen > wrote:


Thanks Josh for looking into this!

On Fri, Apr 5, 2019 at 12:52 PM Josh Elser mailto:els...@apache.org>> wrote:

I can't do this right now, but I've asked Infra to give me the
karma I
need to remove this person.

If a moderator of user@phoenix is watching this, they can use
the tool
on Whimsy[1] to remove this subscriber.

- Josh

[1] https://whimsy.apache.org/committers/moderationhelper.cgi

On 4/5/19 3:47 PM, Josh Elser wrote:
 > Not just you. I have an email filter set up for folks like
this :)
 >
 > A moderator should be able to force a removal. Trying it now
to see if
 > I'm a moderator (I don't think I am, but might be able to add
myself as
 > one).
 >
 > On 4/4/19 7:15 PM, William Shen wrote:
 >> I kept getting this every time I send to the users list. Can
we force
 >> remove the subscriber (martin.pernollet-...@sgcib.com

 >> >) ? Or is this only
happening
 >> to me?
 >>
 >> Thanks
 >>
 >> - Will
 >>
 >> -- Forwarded message -
 >> From: mailto:postmas...@sgcib.com>
>>
 >> Date: Thu, Apr 4, 2019 at 12:17 PM
 >> Subject: DELIVERY FAILURE: Error transferring to
 >> QCMBSJ601.HERMES.SI.SOCGEN; Maximum hop count exceeded. Message
 >> probably in a routing loop.
 >> To: mailto:wills...@marinsoftware.com>
>>
 >>
 >>
 >> Your message
 >>
 >>    Subject: Using Hint with PhoenixRDD
 >>
 >> was not delivered to:
 >>
 >> martin.pernollet-...@sgcib.com

>
 >>
 >> because:
 >>
 >>    Error transferring to QCMBSJ601.HERMES.SI.SOCGEN; Maximum
hop count
 >> exceeded.  Message probably in a routing loop.
 >>
 >>
 >>
 >>
 >> -- Forwarded message --
 >> From: wills...@marinsoftware.com

>
 >> To: user@phoenix.apache.org 
>
 >> Cc:
 >> Bcc:
 >> Date: Thu, 4 Apr 2019 12:16:48 -0700
 >> Subject: Using Hint with PhoenixRDD
 >> Hi all,
 >>
 >> Do we have any way of passing in hints when querying Phoenix
using
 >> PhoenixRDD in Spark? I reviewed the implementation of
PhoenixRDD
 >> and PhoenixRecordWritable, but was not able to find an
obvious way to
 >> do so. Is it supported?
 >>
 >> Thanks in advance!
 >>
 >> - Will
 >>
 >>

*
 >> This message and any attachments (the “message”) are
confidential,
 >> intended solely for the addressee(s), and may contain legally
 >> privileged information.
 >> Any unauthorised use or dissemination is prohibited.
 >> If you are not the intended recipient please notify us
immediately by
 >> telephoning or e-mailing the sender.
 >> You should not copy this e-mail or use it for any purpose
nor disclose
 >> its contents to any other person.
 >> E-mails are susceptible to alteration. Neither SOCIETE
GENERALE nor
 >> any of its subsidiaries or affiliates shall be liable for
the message
 >> if altered, changed or falsified.
 >> You agree to receive all communications by email or through
dedicated
 >> websites.
 >> Please visit
https://cib.societegenerale.com/en/market-regulation/ for
 >> important information on MiFID, SFTR and with respect to
derivative
 >> products.
 >>
 >> Ce message et toutes les pieces jointes sont confidentiels et
 >> susceptibl

Re: PQS + Kerberos problems

2019-05-28 Thread Josh Elser 

Make sure you have authorization set up correctly between PQS and HBase.

Specifically, you must have the appropriate Hadoop proxyuser rules set 
up in core-site.xml so that HBase will allow PQS to impersonate the PQS 
end-user.


On 5/14/19 11:04 AM, Aleksandr Saraseka wrote:

Hello, I have HBase + PQS 4.14.1
If I'm trying to connect by think client - everything works, but if I'm 
using thin client in PQS logs I can see continuous INFO messages
2019-05-14 13:53:58,701 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=10, retries=35, started=48292 ms ago, cancelled=false, msg=

...
2019-05-14 14:18:41,446 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=33, retries=35, started=510325 ms ago, cancelled=false, msg=
2019-05-14 14:19:01,489 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=34, retries=35, started=530368 ms ago, cancelled=false, msg=

...
2019-05-14 14:18:41,446 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=33, retries=35, started=510325 ms ago, cancelled=false, msg=
2019-05-14 14:19:01,489 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=34, retries=35, started=530368 ms ago, cancelled=false, msg=
2019-05-14 14:19:50,139 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=10, retries=35, started=48480 ms ago, cancelled=false, msg=row 
'SYSTEM:CATALOG,,' on table 'hbase:meta' at 
region=hbase:meta,,1.1588230740, hostname=datanode-001.fqdn.com 
,60020,1557323271824, seqNum=0
2019-05-14 14:20:10,333 INFO 
org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, 
tries=11, retries=35, started=68676 ms ago, cancelled=false, msg=row 
'SYSTEM:CATALOG,,' on table 'hbase:meta' at 
region=hbase:meta,,1.1588230740, hostname=datanode-001.fqdn.com 
,60020,1557323271824, seqNum=0


*Hbase security logs:*
2019-05-14 14:42:19,524 INFO 
SecurityLogger.org.apache.hadoop.hbase.Server: Auth successful for 
HTTP/phoenix-queryserver-fqdn@realm.com 
 (auth:KERBEROS)
2019-05-14 14:42:19,524 INFO 
SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 
10.252.16.253 port: 41040 with version info: version: "1.2.0-cdh5.14.2" 
url: 
"file:///data/jenkins/workspace/generic-binary-tarball-and-maven-deploy/CDH5.14.2-Packaging-HBase-2018-03-27_13-15-05/hbase-1.2.0-cdh5.14.2" 
revision: "Unknown" user: "jenkins" date: "Tue Mar 27 13:31:54 PDT 2018" 
src_checksum: "05e6e90e06dd7796f56067208a9bf2aa"
2019-05-14 14:42:29,634 INFO 
SecurityLogger.org.apache.hadoop.hbase.Server: Auth successful for 
HTTP/phoenix-queryserver-fqdn@realm.com 
 (auth:KERBEROS)
2019-05-14 14:42:29,635 INFO 
SecurityLogger.org.apache.hadoop.hbase.Server: Connection from 
10.252.16.253 port: 41046 with version info: version: "1.2.0-cdh5.14.2" 
url: 
"file:///data/jenkins/workspace/generic-binary-tarball-and-maven-deploy/CDH5.14.2-Packaging-HBase-2018-03-27_13-15-05/hbase-1.2.0-cdh5.14.2" 
revision: "Unknown" user: "jenkins" date: "Tue Mar 27 13:31:54 PDT 2018" 
src_checksum: "05e6e90e06dd7796f56067208a9bf2aa"



*thin client logs:*
19/05/14 14:10:08 DEBUG execchain.MainClientExec: Proxy auth state: 
UNCHALLENGED

19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> POST / HTTP/1.1
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Content-Length: 137
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Content-Type: 
application/octet-stream
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Host: 
host-fqdn.com:8765 
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Connection: 
Keep-Alive
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> User-Agent: 
Apache-HttpClient/4.5.2 (Java/1.8.0_161)
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> 
Accept-Encoding: gzip,deflate
19/05/14 14:10:08 DEBUG http.headers: http-outgoing-0 >> Authorization: 
Negotiate 
YIICtQYGKwYBBQUCoIICqTCCAqWgDTALBgkqhkiG9xIBAgKhBAMCAXaiggKMBIICiGCCAoQGCSqGSIb3EgECAgEAboICczCCAm+gAwIBBaEDAgEOogcDBQAgo4IBgGGCAXwwggF4oAMCAQWhEBsOREFUQVNZUy5DRi5XVEaiQDA+oAMCAQChNzA1GwRIVFRQGy1kYXRhc3lzLXNlY3VyZS1odWUwMDEtc3RnLmMuY2Ytc3RhZ2UuaW50ZXJuYWyjggEbMIIBF6ADAgESoQMCAQOiggEJBIIBBTmyywjx8OcQBfIt2AccWAVPKmyf/UM5lDgRUs9cUixE35x6wl9EoIX6XTsw2hO4ESCt1fFrt4XU3iLadKSM2HtR1Zlp/Q4rw/sATQe2DOEQUquVg548kkvZ9c/M1PrigwZnv28Ew7CZd8FWtTB4+PFPMzbkBBXAI2pMAYDAvvUsrUXlgWBTdrg118uN07BApmK3qCB7BAjIJzTONxFmxXAx+PHlXAOiWMGxsi8V421411VCclGFG0txoSkf6aDdhzcZdQZhuHaL73+JfUXvS8j32cadLjhi1CKtAC8ddfG6NfBFciYNNa8khybXrxd/KsWb6TRAgV9EsZKkCNNOMga09Vv/7qSB1TCB0qADAgESooHKBIHHn1c00ERMjywGVzN7mBqq/2Kc+Is+Oxgs+L4t8A5uyx7/m9axITKA5Zr/wC2jLcVGH+CHEbcgb4zc9hCRIM7RS7MTHINGrh5oZAO6KS/LlrlfYYWuZNQEGuyToStLvV6UyuSs8T/DU9+l6+Y4jx5paoGx9/3Fuliy/RKTYWT4opWH9F8aVA4WIPJbLEOp+LNLmMjko8nzoeLEBkl7OLDcUTsbgYVy7WfnKnq05SKxZaLn