[jira] [Commented] (KUDU-2267) Client may see negotiation failure when talks to master followers with only self signed cert
[
https://issues.apache.org/jira/browse/KUDU-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340406#comment-16340406
]
Hao Hao commented on KUDU-2267:
---
Fixed in 8138602ea and c497c69bb.
> Client may see negotiation failure when talks to master followers with only
> self signed cert
> -
>
> Key: KUDU-2267
> URL: https://issues.apache.org/jira/browse/KUDU-2267
> Project: Kudu
> Issue Type: Improvement
> Components: client
>Affects Versions: 1.6.0
>Reporter: Hao Hao
>Priority: Major
>
> Currently, if a master has never been a leader from the very start of the
> cluster, it has just self-signed cert. And if a client does not have valid
> Kerberos credential but only authenticated token, then the client may see
> {{org.apache.kudu.client.NonRecoverableException: Server requires Kerberos,
> but this client is not authenticated}} error when trying to connect to master
> followers. Since in that case SASL authentication type is chosen instead of
> token for authentication.
> It is safe to ignore this error, as long as client is able to connect to
> master leader. However, for a long term fix, masters should probably attempt
> to get a signed cert from the leader.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KUDU-2267) Client may see negotiation failure when talks to master followers with only self signed cert
[
https://issues.apache.org/jira/browse/KUDU-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao resolved KUDU-2267.
---
Resolution: Fixed
Fix Version/s: 1.7.0
Target Version/s: 1.7.0
> Client may see negotiation failure when talks to master followers with only
> self signed cert
> -
>
> Key: KUDU-2267
> URL: https://issues.apache.org/jira/browse/KUDU-2267
> Project: Kudu
> Issue Type: Improvement
> Components: client
>Affects Versions: 1.6.0
>Reporter: Hao Hao
>Priority: Major
> Fix For: 1.7.0
>
>
> Currently, if a master has never been a leader from the very start of the
> cluster, it has just self-signed cert. And if a client does not have valid
> Kerberos credential but only authenticated token, then the client may see
> {{org.apache.kudu.client.NonRecoverableException: Server requires Kerberos,
> but this client is not authenticated}} error when trying to connect to master
> followers. Since in that case SASL authentication type is chosen instead of
> token for authentication.
> It is safe to ignore this error, as long as client is able to connect to
> master leader. However, for a long term fix, masters should probably attempt
> to get a signed cert from the leader.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KUDU-2265) A non-leader master uses self-signed server TLS cert if it hasn't ever run as a leader
[
https://issues.apache.org/jira/browse/KUDU-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao updated KUDU-2265:
--
Resolution: Fixed
Fix Version/s: 1.7.0
Status: Resolved (was: In Review)
> A non-leader master uses self-signed server TLS cert if it hasn't ever run as
> a leader
> --
>
> Key: KUDU-2265
> URL: https://issues.apache.org/jira/browse/KUDU-2265
> Project: Kudu
> Issue Type: Improvement
> Components: master, security
>Affects Versions: 1.3.0, 1.3.1, 1.4.0, 1.6.0, 1.50
>Reporter: Alexey Serbin
>Assignee: Alexey Serbin
>Priority: Major
> Fix For: 1.7.0
>
>
> As it's currently implemented, master process replaces its auto-generated
> self-signed server TLS certificate with CA-signed one only when it becomes a
> leader (see {{{color:#33}CatalogManager::InitCertAuthorityWith(){color}}}
> method; it's invoked only from \{{CatalogManager::InitCertAuthority()}}, that
> is invoked only from \{{CatalogManager::PrepareForLeadershipTask()}}).
>
> In case of just one Raft election from the start (which is pretty common,
> BTW), non-leader masters run without CA-signed server certificate for a long
> time in case of a multi-master configuration. That causes clients to not use
> their authn tokens for authentication while connecting to those non-leader
> masters. In case of Spark applications where executors do not have Kerberos
> credentials (the common case), application logs are polluted with messages
> like below:
> {noformat}
> org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but
> this client is not authenticated (kinit)
> at org.apache.kudu.client.Negotiator.evaluateChallenge(Negotiator.java:705)
> at org.apache.kudu.client.Negotiator.sendSaslInitiate(Negotiator.java:581)
> ...
> Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KUDU-2265) A non-leader master uses self-signed server TLS cert if it hasn't ever run as a leader
[
https://issues.apache.org/jira/browse/KUDU-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340402#comment-16340402
]
Hao Hao commented on KUDU-2265:
---
Fixed in c497c69bb.
> A non-leader master uses self-signed server TLS cert if it hasn't ever run as
> a leader
> --
>
> Key: KUDU-2265
> URL: https://issues.apache.org/jira/browse/KUDU-2265
> Project: Kudu
> Issue Type: Improvement
> Components: master, security
>Affects Versions: 1.3.0, 1.3.1, 1.4.0, 1.6.0, 1.50
>Reporter: Alexey Serbin
>Assignee: Alexey Serbin
>Priority: Major
>
> As it's currently implemented, master process replaces its auto-generated
> self-signed server TLS certificate with CA-signed one only when it becomes a
> leader (see {{{color:#33}CatalogManager::InitCertAuthorityWith(){color}}}
> method; it's invoked only from \{{CatalogManager::InitCertAuthority()}}, that
> is invoked only from \{{CatalogManager::PrepareForLeadershipTask()}}).
>
> In case of just one Raft election from the start (which is pretty common,
> BTW), non-leader masters run without CA-signed server certificate for a long
> time in case of a multi-master configuration. That causes clients to not use
> their authn tokens for authentication while connecting to those non-leader
> masters. In case of Spark applications where executors do not have Kerberos
> credentials (the common case), application logs are polluted with messages
> like below:
> {noformat}
> org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but
> this client is not authenticated (kinit)
> at org.apache.kudu.client.Negotiator.evaluateChallenge(Negotiator.java:705)
> at org.apache.kudu.client.Negotiator.sendSaslInitiate(Negotiator.java:581)
> ...
> Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KUDU-2262) Java client does not retry if no master is a leader
[ https://issues.apache.org/jira/browse/KUDU-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340401#comment-16340401 ] Hao Hao commented on KUDU-2262: --- Fixed in 8138602ea. > Java client does not retry if no master is a leader > --- > > Key: KUDU-2262 > URL: https://issues.apache.org/jira/browse/KUDU-2262 > Project: Kudu > Issue Type: Bug > Components: java >Affects Versions: 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Todd Lipcon >Assignee: Hao Hao >Priority: Major > Fix For: 1.7.0 > > > In a test case I tried to restart the masters and then start a new client to > connect to the cluster. This caused the client to fail because the masters > were in the process of a leader election. > It probably would make more sense for the client to retry a certain number of > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KUDU-2262) Java client does not retry if no master is a leader
[ https://issues.apache.org/jira/browse/KUDU-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Hao resolved KUDU-2262. --- Resolution: Fixed Fix Version/s: 1.7.0 > Java client does not retry if no master is a leader > --- > > Key: KUDU-2262 > URL: https://issues.apache.org/jira/browse/KUDU-2262 > Project: Kudu > Issue Type: Bug > Components: java >Affects Versions: 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Todd Lipcon >Assignee: Hao Hao >Priority: Major > Fix For: 1.7.0 > > > In a test case I tried to restart the masters and then start a new client to > connect to the cluster. This caused the client to fail because the masters > were in the process of a leader election. > It probably would make more sense for the client to retry a certain number of > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KUDU-2262) Java client does not retry if no master is a leader
[ https://issues.apache.org/jira/browse/KUDU-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Hao updated KUDU-2262: -- Affects Version/s: (was: 1.7.0) > Java client does not retry if no master is a leader > --- > > Key: KUDU-2262 > URL: https://issues.apache.org/jira/browse/KUDU-2262 > Project: Kudu > Issue Type: Bug > Components: java >Affects Versions: 1.3.0, 1.4.0, 1.5.0, 1.6.0 >Reporter: Todd Lipcon >Assignee: Hao Hao >Priority: Major > > In a test case I tried to restart the masters and then start a new client to > connect to the cluster. This caused the client to fail because the masters > were in the process of a leader election. > It probably would make more sense for the client to retry a certain number of > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KUDU-2262) Java client does not retry if no master is a leader
[ https://issues.apache.org/jira/browse/KUDU-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Hao updated KUDU-2262: -- Affects Version/s: 1.3.0 1.4.0 1.5.0 1.6.0 > Java client does not retry if no master is a leader > --- > > Key: KUDU-2262 > URL: https://issues.apache.org/jira/browse/KUDU-2262 > Project: Kudu > Issue Type: Bug > Components: java >Affects Versions: 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0 >Reporter: Todd Lipcon >Assignee: Hao Hao >Priority: Major > > In a test case I tried to restart the masters and then start a new client to > connect to the cluster. This caused the client to fail because the masters > were in the process of a leader election. > It probably would make more sense for the client to retry a certain number of > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KUDU-2243) CFile Reader improvements
[
https://issues.apache.org/jira/browse/KUDU-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339682#comment-16339682
]
Dan Burkert commented on KUDU-2243:
---
We discussed the concept renaming on slack and came to the following
conclusions:
CFile should be renamed to Chunk:
* CFile has connotations that it's a file, but in reality a CFile is 1:1 with
the fs::BlockManager Block.
* CFile maps very closely to Parquet's Column Chunk Abstraction ('Column
chunk: A chunk of the data for a particular column')
* We'd therefore have column chunks, ad-hoc index chunks, bloom chunks, and
delta chunks
cfile block/cblock should be renamed to page:
* As the unit of encoding and compression, and the smallest indivisible
on-disk container, it maps very well to the classical database concept of a
page.
* It maps well to Parquet's concept of a page ('Page: Column chunks are
divided up into pages. A page is conceptually an indivisible unit (in terms of
compression and encoding). There can be multiple page types which is
interleaved in a column chunk.')
The current fs block manager block abstraction will remain, to which the
'block' term will unambiguously refer.
> CFile Reader improvements
> -
>
> Key: KUDU-2243
> URL: https://issues.apache.org/jira/browse/KUDU-2243
> Project: Kudu
> Issue Type: Improvement
> Components: cfile
>Affects Versions: 1.6.0
>Reporter: Dan Burkert
>Priority: Major
>
> I've done a pretty thorough review of all the CFile reader code over the last
> few days in order to make a targeted bug fix, and I've got some ideas for how
> we can simplify it. I'd like to get others thoughts.
> * To reduce confusion between CFile data blocks and FS manager blocks, I
> think we should change all references in code and docs of CFile data blocks
> to 'cblock'.
> * Much of the complexity of the CFileIterator is due to it's complex public
> API, which requires separate {{Seek(idx) -> Prepare(nrows) -> Scan(output
> buf, predicates)}} calls. Additionally, the Prepare step can materialize
> many blocks, which then need to be put in a queue. I think all of this could
> be simplified by changing the API to be {{Seek(idx) -> Scan(nrows, output
> buf, predicates)}}, and have the CFile iterator only cache the
> most-recently-materialized block (instead of the queue). For really big scan
> batches, this will change the internal scan/materialize pattern from
> materializing all cblocks up front then copying, to materializing and copying
> of cblocks being interleaved. Since in most cases cblocks are usually much
> bigger (256kib) than scan batches (100 cells), I think it won't actually lead
> to measurably different behavior.
> * {{QueueCurrentDataBlock}} and {{ReadCurrentDataBlock}} should drop
> {{Current}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
