[jira] [Commented] (KUDU-1860) ksck doesn't identify tablets that are evicted but still in config

2017-02-01 Thread Jean-Daniel Cryans (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849314#comment-15849314
 ] 

Jean-Daniel Cryans commented on KUDU-1860:
--

bq. To clarify: it's "evicted" meaning that there is a pending configuration 
that removes it, but the pending configuration is not yet committed?

Yes.

bq. We don't currently centralize the pending config to the master IIRC, but we 
could consider doing so, or fetching it from tservers during ksck (which might 
be less error-prone)

Trying to reconcile the views of all the replicas could be tricky, maybe giving 
some false positives since it wouldn't be atomic, but it seems better than not 
showing bad tablets. We could also consider doing some health checks like 
sending some dummy message that the config has to replicate, and if that works 
then consider that tablet ok.

> ksck doesn't identify tablets that are evicted but still in config
> --
>
> Key: KUDU-1860
> URL: https://issues.apache.org/jira/browse/KUDU-1860
> Project: Kudu
>  Issue Type: Bug
>  Components: util
>Affects Versions: 1.2.0
>Reporter: Jean-Daniel Cryans
>Priority: Critical
>
> As reported by a user on Slack, ksck can give you a wrong output such as:
> {noformat}
>   ca199fafca544df2a1b2a01be9d5266d (server1:7250): RUNNING [LEADER]
>   a077957f627c4758ab5a989aca8a1ca8 (server2:7250): RUNNING
>   5c09a555c205482b8131f15b2c249ec6 (server3:7250): bad state
> State:   NOT_STARTED
> Data state:  TABLET_DATA_TOMBSTONED
> Last status: Tablet initializing...
> {noformat}
> The problem is that server2 was already evicted out of the configuration 
> (based on reading the logs) but it wasn't committed in the config (which 
> contains server 1 and 3) since there's really only 1 server left out of 3.
> Ideally ksck should try to see what each server thinks the configuration is 
> and see if there's a difference from what's in the master. As it is, it looks 
> like we're missing 1 replica but in reality this is a broken tablet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1860) ksck doesn't identify tablets that are evicted but still in config

2017-02-01 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849299#comment-15849299
 ] 

Todd Lipcon commented on KUDU-1860:
---

To clarify: it's "evicted" meaning that there is a pending configuration that 
removes it, but the pending configuration is not yet committed?

Agreed it would be great to show pending config information here. We don't 
currently centralize the pending config to the master IIRC, but we could 
consider doing so, or fetching it from tservers during ksck (which might be 
less error-prone)

> ksck doesn't identify tablets that are evicted but still in config
> --
>
> Key: KUDU-1860
> URL: https://issues.apache.org/jira/browse/KUDU-1860
> Project: Kudu
>  Issue Type: Bug
>  Components: util
>Affects Versions: 1.2.0
>Reporter: Jean-Daniel Cryans
>Priority: Critical
>
> As reported by a user on Slack, ksck can give you a wrong output such as:
> {noformat}
>   ca199fafca544df2a1b2a01be9d5266d (server1:7250): RUNNING [LEADER]
>   a077957f627c4758ab5a989aca8a1ca8 (server2:7250): RUNNING
>   5c09a555c205482b8131f15b2c249ec6 (server3:7250): bad state
> State:   NOT_STARTED
> Data state:  TABLET_DATA_TOMBSTONED
> Last status: Tablet initializing...
> {noformat}
> The problem is that server2 was already evicted out of the configuration 
> (based on reading the logs) but it wasn't committed in the config (which 
> contains server 1 and 3) since there's really only 1 server left out of 3.
> Ideally ksck should try to see what each server thinks the configuration is 
> and see if there's a difference from what's in the master. As it is, it looks 
> like we're missing 1 replica but in reality this is a broken tablet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Created] (KUDU-1860) ksck doesn't identify tablets that are evicted but still in config

2017-02-01 Thread Jean-Daniel Cryans (JIRA)
Jean-Daniel Cryans created KUDU-1860:


 Summary: ksck doesn't identify tablets that are evicted but still 
in config
 Key: KUDU-1860
 URL: https://issues.apache.org/jira/browse/KUDU-1860
 Project: Kudu
  Issue Type: Bug
  Components: util
Affects Versions: 1.2.0
Reporter: Jean-Daniel Cryans
Priority: Critical


As reported by a user on Slack, ksck can give you a wrong output such as:

{noformat}
  ca199fafca544df2a1b2a01be9d5266d (server1:7250): RUNNING [LEADER]
  a077957f627c4758ab5a989aca8a1ca8 (server2:7250): RUNNING
  5c09a555c205482b8131f15b2c249ec6 (server3:7250): bad state
State:   NOT_STARTED
Data state:  TABLET_DATA_TOMBSTONED
Last status: Tablet initializing...
{noformat}

The problem is that server2 was already evicted out of the configuration (based 
on reading the logs) but it wasn't committed in the config (which contains 
server 1 and 3) since there's really only 1 server left out of 3.

Ideally ksck should try to see what each server thinks the configuration is and 
see if there's a difference from what's in the master. As it is, it looks like 
we're missing 1 replica but in reality this is a broken tablet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Will Berkeley (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848953#comment-15848953
 ] 

Will Berkeley commented on KUDU-1859:
-

That fact does make it pretty hard to justify implementing, given that it 
wouldn't be really easy.

> Allow NULL in IN LIST predicates
> 
>
> Key: KUDU-1859
> URL: https://issues.apache.org/jira/browse/KUDU-1859
> Project: Kudu
>  Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Will Berkeley
>Priority: Minor
>
> Kudu doesn't presently allow NULL values along with ordinary values in IN 
> LIST predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Dan Burkert (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848949#comment-15848949
 ] 

Dan Burkert commented on KUDU-1859:
---

Yah, it effectively ignores it because NULL != NULL, so NULL in an IN list 
matches no rows.

> Allow NULL in IN LIST predicates
> 
>
> Key: KUDU-1859
> URL: https://issues.apache.org/jira/browse/KUDU-1859
> Project: Kudu
>  Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Will Berkeley
>Priority: Minor
>
> Kudu doesn't presently allow NULL values along with ordinary values in IN 
> LIST predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Will Berkeley (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848943#comment-15848943
 ] 

Will Berkeley commented on KUDU-1859:
-

See Alexey's comment in KUDU-1642 for info about psql's handling of NULL in 
list.

I've not got strong feelings either way on this. I just wanted to post it as a 
separate JIRA so it didn't get forgotten about, since it came up a couple of 
times over the lifetime of KUDU-1642.

> Allow NULL in IN LIST predicates
> 
>
> Key: KUDU-1859
> URL: https://issues.apache.org/jira/browse/KUDU-1859
> Project: Kudu
>  Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Will Berkeley
>Priority: Minor
>
> Kudu doesn't presently allow NULL values along with ordinary values in IN 
> LIST predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848937#comment-15848937
 ] 

Todd Lipcon commented on KUDU-1859:
---

what does postgres do with a NULL in an in-list? Does it just ignore it?

> Allow NULL in IN LIST predicates
> 
>
> Key: KUDU-1859
> URL: https://issues.apache.org/jira/browse/KUDU-1859
> Project: Kudu
>  Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Will Berkeley
>Priority: Minor
>
> Kudu doesn't presently allow NULL values along with ordinary values in IN 
> LIST predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Resolved] (KUDU-1642) Add IS NULL predicate type

2017-02-01 Thread Will Berkeley (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Berkeley resolved KUDU-1642.
-
   Resolution: Fixed
Fix Version/s: 1.3.0

Resolved in 431aee53fd269b5699095fb1e7fd5402644fe454. Support for NULL in IN 
LIST predicates was not added. See KUDU-1859.

> Add IS NULL predicate type
> --
>
> Key: KUDU-1642
> URL: https://issues.apache.org/jira/browse/KUDU-1642
> Project: Kudu
>  Issue Type: Sub-task
>  Components: client, tablet
>Reporter: Dan Burkert
>Assignee: Will Berkeley
> Fix For: 1.3.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Dan Burkert (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848914#comment-15848914
 ] 

Dan Burkert commented on KUDU-1859:
---

I don't think we should do this, because it will complicate the implementation. 
 If SQL layers above us want to handle NULL in IN lists they can filter it 
themselves before passing to Kudu.

> Allow NULL in IN LIST predicates
> 
>
> Key: KUDU-1859
> URL: https://issues.apache.org/jira/browse/KUDU-1859
> Project: Kudu
>  Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Will Berkeley
>
> Kudu doesn't presently allow NULL values along with ordinary values in IN 
> LIST predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Created] (KUDU-1859) Allow NULL in IN LIST predicates

2017-02-01 Thread Will Berkeley (JIRA)
Will Berkeley created KUDU-1859:
---

 Summary: Allow NULL in IN LIST predicates
 Key: KUDU-1859
 URL: https://issues.apache.org/jira/browse/KUDU-1859
 Project: Kudu
  Issue Type: Improvement
Affects Versions: 1.2.0
Reporter: Will Berkeley


Kudu doesn't presently allow NULL values along with ordinary values in IN LIST 
predicates. PostgreSQL does, for example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Resolved] (KUDU-1595) Expose IS NOT NULL predicates in client API

2017-02-01 Thread Will Berkeley (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Will Berkeley resolved KUDU-1595.
-
   Resolution: Fixed
Fix Version/s: 1.3.0

Resolved in 431aee53fd269b5699095fb1e7fd5402644fe454

> Expose IS NOT NULL predicates in client API
> ---
>
> Key: KUDU-1595
> URL: https://issues.apache.org/jira/browse/KUDU-1595
> Project: Kudu
>  Issue Type: Sub-task
>  Components: client
>Reporter: Todd Lipcon
>Assignee: Will Berkeley
>  Labels: newbie
> Fix For: 1.3.0
>
>
> It would be helpful to be able to push down IS NOT NULL and IS NULL 
> predicates from query engines into Kudu. We already have much of the 
> implementation internally, but it's not exposed in the API, and would need 
> some tests.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1738) Allow users of C++ client to disable initialization of OpenSSL

2017-02-01 Thread Sailesh Mukil (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848859#comment-15848859
 ] 

Sailesh Mukil commented on KUDU-1738:
-

[~tlipcon] Ah yes, my bad. That makes it more clear. I was just thinking about 
internal and external RPCs. This is a necessity even now then.

> Allow users of C++ client to disable initialization of OpenSSL
> --
>
> Key: KUDU-1738
> URL: https://issues.apache.org/jira/browse/KUDU-1738
> Project: Kudu
>  Issue Type: Bug
>  Components: client, security
>Affects Versions: 1.1.0
>Reporter: Todd Lipcon
>Priority: Blocker
>
> As we start to use OpenSSL from within the client, we now take care of 
> initializing the OpenSSL library. Programs which embed us may have already 
> initialized OpenSSL for their own purposes. In that case, we don't want Kudu 
> to run the initialization a second time.
> We should probably add some kind of option to KuduClientBuilder which 
> disables the SSL initialization. Another thought is that we may want to 
> detect if the SSL library has already been initialized and automatically 
> skip. (it seems this is possible by calling SSL_CTX_new and checking the 
> error code to see if it's not initialized yet)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Updated] (KUDU-1854) Fix links to Cloudera 'beta' docs across all topics

2017-02-01 Thread Ambreen Kazi (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ambreen Kazi updated KUDU-1854:
---
Code Review: https://gerrit.cloudera.org/#/c/5837/

> Fix links to Cloudera 'beta' docs across all topics
> ---
>
> Key: KUDU-1854
> URL: https://issues.apache.org/jira/browse/KUDU-1854
> Project: Kudu
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.2.0
>Reporter: Ambreen Kazi
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1854) Fix links to Cloudera 'beta' docs across all topics

2017-02-01 Thread Ambreen Kazi (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848823#comment-15848823
 ] 

Ambreen Kazi commented on KUDU-1854:


Merged - https://gerrit.cloudera.org/#/c/5837/

> Fix links to Cloudera 'beta' docs across all topics
> ---
>
> Key: KUDU-1854
> URL: https://issues.apache.org/jira/browse/KUDU-1854
> Project: Kudu
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.2.0
>Reporter: Ambreen Kazi
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Created] (KUDU-1858) Record outbound call queuing / sending times

2017-02-01 Thread Henry Robinson (JIRA)
Henry Robinson created KUDU-1858:


 Summary: Record outbound call queuing / sending times
 Key: KUDU-1858
 URL: https://issues.apache.org/jira/browse/KUDU-1858
 Project: Kudu
  Issue Type: Improvement
Reporter: Henry Robinson


{{InboundCall}} does a pretty good job of tracking the time it spends queued, 
being handled and so on. We should consider adding the same kind of 
instrumentation to {{OutboundCall}} so that RPC senders can track the time an 
RPC took being queued for sending, actually sending and then waiting for a 
response. 

This information could then be exposed through the {{RpcController}} so that, 
upon RPC completion (or failure) the measurements could be aggregated.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1858) Record outbound call queuing / sending times

2017-02-01 Thread Henry Robinson (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848803#comment-15848803
 ] 

Henry Robinson commented on KUDU-1858:
--

That might be useful - it's debatable, though, if we would want to communicate 
the sender-side state transitions to the client (for no other reason than that 
adds a tight coupling between server implementation and the client). 

> Record outbound call queuing / sending times
> 
>
> Key: KUDU-1858
> URL: https://issues.apache.org/jira/browse/KUDU-1858
> Project: Kudu
>  Issue Type: Improvement
>Reporter: Henry Robinson
>
> {{InboundCall}} does a pretty good job of tracking the time it spends queued, 
> being handled and so on. We should consider adding the same kind of 
> instrumentation to {{OutboundCall}} so that RPC senders can track the time an 
> RPC took being queued for sending, actually sending and then waiting for a 
> response. 
> This information could then be exposed through the {{RpcController}} so that, 
> upon RPC completion (or failure) the measurements could be aggregated.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1738) Allow users of C++ client to disable initialization of OpenSSL

2017-02-01 Thread Henry Robinson (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848797#comment-15848797
 ] 

Henry Robinson commented on KUDU-1738:
--

Todd's right - there are many different clients of OpenSSL in Impala, and 
there's no likelihood of that number decreasing in the medium term.

> Allow users of C++ client to disable initialization of OpenSSL
> --
>
> Key: KUDU-1738
> URL: https://issues.apache.org/jira/browse/KUDU-1738
> Project: Kudu
>  Issue Type: Bug
>  Components: client, security
>Affects Versions: 1.1.0
>Reporter: Todd Lipcon
>Priority: Blocker
>
> As we start to use OpenSSL from within the client, we now take care of 
> initializing the OpenSSL library. Programs which embed us may have already 
> initialized OpenSSL for their own purposes. In that case, we don't want Kudu 
> to run the initialization a second time.
> We should probably add some kind of option to KuduClientBuilder which 
> disables the SSL initialization. Another thought is that we may want to 
> detect if the SSL library has already been initialized and automatically 
> skip. (it seems this is possible by calling SSL_CTX_new and checking the 
> error code to see if it's not initialized yet)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1858) Record outbound call queuing / sending times

2017-02-01 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848796#comment-15848796
 ] 

Todd Lipcon commented on KUDU-1858:
---

Sounds useful! It also might be interesting to add some kind of 'ACK' from the 
server when a call has been received but not yet processed, but I guess that 
would be a follow-up task.

> Record outbound call queuing / sending times
> 
>
> Key: KUDU-1858
> URL: https://issues.apache.org/jira/browse/KUDU-1858
> Project: Kudu
>  Issue Type: Improvement
>Reporter: Henry Robinson
>
> {{InboundCall}} does a pretty good job of tracking the time it spends queued, 
> being handled and so on. We should consider adding the same kind of 
> instrumentation to {{OutboundCall}} so that RPC senders can track the time an 
> RPC took being queued for sending, actually sending and then waiting for a 
> response. 
> This information could then be exposed through the {{RpcController}} so that, 
> upon RPC completion (or failure) the measurements could be aggregated.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KUDU-1855) Kudu file UNIX permissions are inconsistent

2017-02-01 Thread Todd Lipcon (JIRA)

[ 
https://issues.apache.org/jira/browse/KUDU-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848772#comment-15848772
 ] 

Todd Lipcon commented on KUDU-1855:
---

Ah, OK. Let's consider this a blocker for 1.3, then, since we're adding 
authorization.

> Kudu file UNIX permissions are inconsistent
> ---
>
> Key: KUDU-1855
> URL: https://issues.apache.org/jira/browse/KUDU-1855
> Project: Kudu
>  Issue Type: Bug
>  Components: security
>Affects Versions: 1.3.0
>Reporter: Adar Dembo
>
> Right now the access modes of Kudu files on disk are quite inconsistent. For 
> example:
> {noformat}
> 6942960 drwxr-xr-x   4 kudu kudu   27 Jun 21  2016 
> /data/1/kudu/tablet
> 21501102670 drwxr-xr-x   5 kudu kudu   71 Jan 17 17:42 
> /data/1/kudu/tablet/data
> 21501241274 -rw---   1 kudu kudu  665 Jun 21  2016 
> /data/1/kudu/tablet/data/instance
> 702851   16 drwxr-xr-x   2 kudu kudu12288 Jan 31 14:02 
> /data/1/kudu/tablet/data/tablet-meta
> 702894   12 -rw---   1 kudu kudu 9501 Jan 22 13:38 
> /data/1/kudu/tablet/data/tablet-meta/33cc61001d1442048a588f930d973464
> ...
> 21538687298 -rw---   1 kudu kudu 8109 Jan 12 22:26 
> /data/1/kudu/tablet/data/consensus-meta/83a2f75a88bd48f5a8ec28f8328af481
> 702854  152 drwxr-xr-x   2 kudu kudu   110592 Jan 23 16:59 
> /data/1/kudu/tablet/data/data
> 15456410 -rw-r--r--   1 kudu kudu 10743091200 Jan  9 13:06 
> /data/1/kudu/tablet/data/data/bdf6d87c2ba34d598327b0b9e159a5ea.data
> 7056904 -rw---   1 kudu kudu  990 Jun 21  2016 
> /data/1/kudu/tablet/data/data/block_manager_instance
> 278147  652 -rw-r--r--   1 kudu kudu   660084 Aug 15 00:09 
> /data/1/kudu/tablet/data/data/3917e1185471419786f59ad72bb7ba63.metadata
> 2905870 -rw-r--r--   1 kudu kudu 10746855424 Jan  9 13:05 
> /data/1/kudu/tablet/data/data/3917e1185471419786f59ad72bb7ba63.data
> 291129  672 -rw-r--r--   1 kudu kudu   681669 Aug 15 00:09 
> /data/1/kudu/tablet/data/data/756bd793f0c24188853e096f835ba7b4.metadata
> {noformat}
> I suspect it's due to the difference of "files opened as temp files and 
> renamed into place" and "files opened directly". The former have permissions 
> that are even more restrictive than the umask (022), while the latter adhere 
> to the umask.
> I don't know whether more restrictive or less restrictive modes should be 
> used, but I don't see any reason why all files in Kudu's filesystem layout 
> shouldn't have the same permissions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Updated] (KUDU-1855) Kudu file UNIX permissions are inconsistent

2017-02-01 Thread Todd Lipcon (JIRA)

 [ 
https://issues.apache.org/jira/browse/KUDU-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated KUDU-1855:
--
Priority: Blocker  (was: Major)

> Kudu file UNIX permissions are inconsistent
> ---
>
> Key: KUDU-1855
> URL: https://issues.apache.org/jira/browse/KUDU-1855
> Project: Kudu
>  Issue Type: Bug
>  Components: security
>Affects Versions: 1.3.0
>Reporter: Adar Dembo
>Priority: Blocker
>
> Right now the access modes of Kudu files on disk are quite inconsistent. For 
> example:
> {noformat}
> 6942960 drwxr-xr-x   4 kudu kudu   27 Jun 21  2016 
> /data/1/kudu/tablet
> 21501102670 drwxr-xr-x   5 kudu kudu   71 Jan 17 17:42 
> /data/1/kudu/tablet/data
> 21501241274 -rw---   1 kudu kudu  665 Jun 21  2016 
> /data/1/kudu/tablet/data/instance
> 702851   16 drwxr-xr-x   2 kudu kudu12288 Jan 31 14:02 
> /data/1/kudu/tablet/data/tablet-meta
> 702894   12 -rw---   1 kudu kudu 9501 Jan 22 13:38 
> /data/1/kudu/tablet/data/tablet-meta/33cc61001d1442048a588f930d973464
> ...
> 21538687298 -rw---   1 kudu kudu 8109 Jan 12 22:26 
> /data/1/kudu/tablet/data/consensus-meta/83a2f75a88bd48f5a8ec28f8328af481
> 702854  152 drwxr-xr-x   2 kudu kudu   110592 Jan 23 16:59 
> /data/1/kudu/tablet/data/data
> 15456410 -rw-r--r--   1 kudu kudu 10743091200 Jan  9 13:06 
> /data/1/kudu/tablet/data/data/bdf6d87c2ba34d598327b0b9e159a5ea.data
> 7056904 -rw---   1 kudu kudu  990 Jun 21  2016 
> /data/1/kudu/tablet/data/data/block_manager_instance
> 278147  652 -rw-r--r--   1 kudu kudu   660084 Aug 15 00:09 
> /data/1/kudu/tablet/data/data/3917e1185471419786f59ad72bb7ba63.metadata
> 2905870 -rw-r--r--   1 kudu kudu 10746855424 Jan  9 13:05 
> /data/1/kudu/tablet/data/data/3917e1185471419786f59ad72bb7ba63.data
> 291129  672 -rw-r--r--   1 kudu kudu   681669 Aug 15 00:09 
> /data/1/kudu/tablet/data/data/756bd793f0c24188853e096f835ba7b4.metadata
> {noformat}
> I suspect it's due to the difference of "files opened as temp files and 
> renamed into place" and "files opened directly". The former have permissions 
> that are even more restrictive than the umask (022), while the latter adhere 
> to the umask.
> I don't know whether more restrictive or less restrictive modes should be 
> used, but I don't see any reason why all files in Kudu's filesystem layout 
> shouldn't have the same permissions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)