[jira] [Commented] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[
https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986043#comment-15986043
]
Hadoop QA commented on SENTRY-1400:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865277/SENTRY-1400.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2581/console
This message is automatically generated.
> [Flaky test] TestSentryWebServerWithSSL times out
> -
>
> Key: SENTRY-1400
> URL: https://issues.apache.org/jira/browse/SENTRY-1400
> Project: Sentry
> Issue Type: Test
>Affects Versions: sentry-ha-redesign
>Reporter: Sravya Tirukkovalur
>Assignee: Na Li
> Labels: test
> Attachments: SENTRY-1400.001-sentry-ha-redesign.patch, Sentry-1649
> Patch30 Test fail build 2569 a.zip
>
>
> I atleast saw two precommit jobs which failed due to this:
> https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console
> https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console
> [~lili] any idea whats going on?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[
https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986016#comment-15986016
]
Hadoop QA commented on SENTRY-1649:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865261/SENTRY-1649.031-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2580/console
This message is automatically generated.
> Initialize HMSFollower when sentry server actually starts
> -
>
> Key: SENTRY-1649
> URL: https://issues.apache.org/jira/browse/SENTRY-1649
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Na Li
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1649.015-sentry-ha-redesign.patch,
> SENTRY-1649.022-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.025-sentry-ha-redesign.patch,
> SENTRY-1649.028-sentry-ha-redesign.patch,
> SENTRY-1649.029-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.031-sentry-ha-redesign.patch,
> SENTRY-1649.031-sentry-ha-redesign.patch
>
>
> Now HMSFollower has been initialized at the constructor of SentryService. It
> would be better to initialize it when the service starts, e.g runServer().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[ https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1400: -- Labels: test (was: ) Affects Version/s: sentry-ha-redesign Status: Patch Available (was: Open) > [Flaky test] TestSentryWebServerWithSSL times out > - > > Key: SENTRY-1400 > URL: https://issues.apache.org/jira/browse/SENTRY-1400 > Project: Sentry > Issue Type: Test >Affects Versions: sentry-ha-redesign >Reporter: Sravya Tirukkovalur >Assignee: Na Li > Labels: test > Attachments: SENTRY-1400.001-sentry-ha-redesign.patch, Sentry-1649 > Patch30 Test fail build 2569 a.zip > > > I atleast saw two precommit jobs which failed due to this: > https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console > https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console > [~lili] any idea whats going on? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[ https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1400: -- Attachment: SENTRY-1400.001-sentry-ha-redesign.patch v1.add debug log for key store path at SentryServiceIntegrationBase.setupConf() > [Flaky test] TestSentryWebServerWithSSL times out > - > > Key: SENTRY-1400 > URL: https://issues.apache.org/jira/browse/SENTRY-1400 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Na Li > Attachments: SENTRY-1400.001-sentry-ha-redesign.patch, Sentry-1649 > Patch30 Test fail build 2569 a.zip > > > I atleast saw two precommit jobs which failed due to this: > https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console > https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console > [~lili] any idea whats going on? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
[
https://issues.apache.org/jira/browse/SENTRY-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985936#comment-15985936
]
Hadoop QA commented on SENTRY-1727:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865244/SENTRY-1727.00-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2579/console
This message is automatically generated.
> HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
> -
>
> Key: SENTRY-1727
> URL: https://issues.apache.org/jira/browse/SENTRY-1727
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: hmspaths-debug.patch,
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End-output.txt,
> SENTRY-1727.00-sentry-ha-redesign.patch
>
>
> {{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
> TPathsDump. Internally it calls {{cloneToEntry}} to construct
> {{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
> objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
> properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.031-sentry-ha-redesign.patch v31 again to trigger build > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.015-sentry-ha-redesign.patch, > SENTRY-1649.022-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.025-sentry-ha-redesign.patch, > SENTRY-1649.028-sentry-ha-redesign.patch, > SENTRY-1649.029-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1726) sql changes to store last notification-id processed
[
https://issues.apache.org/jira/browse/SENTRY-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985872#comment-15985872
]
Hadoop QA commented on SENTRY-1726:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865230/SENTRY-1726.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2578/console
This message is automatically generated.
> sql changes to store last notification-id processed
> ---
>
> Key: SENTRY-1726
> URL: https://issues.apache.org/jira/browse/SENTRY-1726
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: kalyan kumar kalvagadda
>Assignee: kalyan kumar kalvagadda
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1726.001-sentry-ha-redesign.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
[
https://issues.apache.org/jira/browse/SENTRY-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu reassigned SENTRY-1727:
-
Assignee: Lei (Eddy) Xu
> HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
> -
>
> Key: SENTRY-1727
> URL: https://issues.apache.org/jira/browse/SENTRY-1727
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: hmspaths-debug.patch,
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End-output.txt,
> SENTRY-1727.00-sentry-ha-redesign.patch
>
>
> {{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
> TPathsDump. Internally it calls {{cloneToEntry}} to construct
> {{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
> objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
> properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
[
https://issues.apache.org/jira/browse/SENTRY-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated SENTRY-1727:
--
Attachment: SENTRY-1727.00-sentry-ha-redesign.patch
Upload the patch
> HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
> -
>
> Key: SENTRY-1727
> URL: https://issues.apache.org/jira/browse/SENTRY-1727
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Hao Hao
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: hmspaths-debug.patch,
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End-output.txt,
> SENTRY-1727.00-sentry-ha-redesign.patch
>
>
> {{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
> TPathsDump. Internally it calls {{cloneToEntry}} to construct
> {{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
> objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
> properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
[
https://issues.apache.org/jira/browse/SENTRY-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated SENTRY-1727:
--
Status: Patch Available (was: Open)
> HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
> -
>
> Key: SENTRY-1727
> URL: https://issues.apache.org/jira/browse/SENTRY-1727
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Lei (Eddy) Xu
> Fix For: 1.8.0, sentry-ha-redesign
>
> Attachments: hmspaths-debug.patch,
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End-output.txt,
> SENTRY-1727.00-sentry-ha-redesign.patch
>
>
> {{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
> TPathsDump. Internally it calls {{cloneToEntry}} to construct
> {{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
> objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
> properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
[
https://issues.apache.org/jira/browse/SENTRY-1727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao updated SENTRY-1727:
Affects Version/s: sentry-ha-redesign
1.8.0
> HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
> -
>
> Key: SENTRY-1727
> URL: https://issues.apache.org/jira/browse/SENTRY-1727
> Project: Sentry
> Issue Type: Bug
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Hao Hao
> Fix For: 1.8.0, sentry-ha-redesign
>
>
> {{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
> TPathsDump. Internally it calls {{cloneToEntry}} to construct
> {{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
> objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
> properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (SENTRY-1727) HMSPathsDumper.cloneToEntry should set authzObjToEntries properly
Hao Hao created SENTRY-1727:
---
Summary: HMSPathsDumper.cloneToEntry should set authzObjToEntries
properly
Key: SENTRY-1727
URL: https://issues.apache.org/jira/browse/SENTRY-1727
Project: Sentry
Issue Type: Bug
Reporter: Hao Hao
Fix For: 1.8.0, sentry-ha-redesign
{{HMSPathsDumper.initializeFromDump}} is trying to clone a new HMSPaths from
TPathsDump. Internally it calls {{cloneToEntry}} to construct
{{HMSPaths.authzObjToEntries}}, which is representation of hive authorized
objects to path entries mapping. It appears to be {{cloneToEntry}} fail to
properly construct {{authzObjToEntries}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1726) sql changes to store last notification-id processed
[ https://issues.apache.org/jira/browse/SENTRY-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kalyan kumar kalvagadda updated SENTRY-1726: Attachment: SENTRY-1726.001-sentry-ha-redesign.patch > sql changes to store last notification-id processed > --- > > Key: SENTRY-1726 > URL: https://issues.apache.org/jira/browse/SENTRY-1726 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1726.001-sentry-ha-redesign.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1726) sql changes to store last notification-id processed
[ https://issues.apache.org/jira/browse/SENTRY-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kalyan kumar kalvagadda updated SENTRY-1726: Fix Version/s: sentry-ha-redesign Status: Patch Available (was: In Progress) Added sql changes for DB2, mysql, oracle, postgres and derby > sql changes to store last notification-id processed > --- > > Key: SENTRY-1726 > URL: https://issues.apache.org/jira/browse/SENTRY-1726 > Project: Sentry > Issue Type: Improvement > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1726.001-sentry-ha-redesign.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (SENTRY-1726) sql changes to store last notification-id processed
kalyan kumar kalvagadda created SENTRY-1726: --- Summary: sql changes to store last notification-id processed Key: SENTRY-1726 URL: https://issues.apache.org/jira/browse/SENTRY-1726 Project: Sentry Issue Type: Improvement Components: Sentry Affects Versions: sentry-ha-redesign Reporter: kalyan kumar kalvagadda Assignee: kalyan kumar kalvagadda -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[ https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1400: -- Attachment: Sentry-1649 Patch30 Test fail build 2569 a.zip console output for this case is attached: Sentry-1649 Patch30 Test fail build 2569 a.zip > [Flaky test] TestSentryWebServerWithSSL times out > - > > Key: SENTRY-1400 > URL: https://issues.apache.org/jira/browse/SENTRY-1400 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Na Li > Attachments: Sentry-1649 Patch30 Test fail build 2569 a.zip > > > I atleast saw two precommit jobs which failed due to this: > https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console > https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console > [~lili] any idea whats going on? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[ https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985577#comment-15985577 ] Na Li commented on SENTRY-1400: --- In the console output, I can see most paths are "/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build@2", but plugin is under "/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build%402" Examples: "2017-04-26 00:54:34,026 (main) [DEBUG - net.sf.ehcache.config.DiskStoreConfiguration.translatePath(DiskStoreConfiguration.java:118)] Disk Store Path: /home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build@2/sentry-provider/sentry-provider-db/target/1493168072479/cache" "Debug is true storeKey true useTicketCache true useKeyTab true doNotPrompt true ticketCache is null isInitiator false KeyTab is /home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build@2/sentry-provider/sentry-provider-db/target/server.keytab refreshKrb5Config is true principal is sentry/localh...@example.com tryFirstPass is false useFirstPass is false storePass is false clearPass is false" "2017-04-26 00:54:33,974 (main) [DEBUG - net.sf.ehcache.config.ConfigurationFactory.parseConfiguration(ConfigurationFactory.java:96)] Configuring ehcache from URL: jar:file:/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build%402/maven-repo/org/apache/directory/server/apacheds-core-api/2.0.0-M15/apacheds-core-api-2.0.0-M15.jar!/directory-cacheservice.xml" But > [Flaky test] TestSentryWebServerWithSSL times out > - > > Key: SENTRY-1400 > URL: https://issues.apache.org/jira/browse/SENTRY-1400 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Na Li > > I atleast saw two precommit jobs which failed due to this: > https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console > https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console > [~lili] any idea whats going on? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[
https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985528#comment-15985528
]
Na Li commented on SENTRY-1400:
---
I suspect the cause of the issue is that
1) When the keystore path is set in SentryServiceIntegrationBase.setupConf()
line 186, the path is relative path.
"conf.set(ServerConfig.SENTRY_WEB_SSL_KEYSTORE_PATH,
Resources.getResource("keystore.jks").getPath());"
2) When SentryWebServer.start() is called, the key store path is used to get
keystore.jks, the absolute path of the key store is wrong because the root
directory is not the root where the keystore.jks is. In other words,
"/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build%402" is not where
the directory in step 1) is.
the build cannot find
"/home/jenkins/jenkins-slave/workspace/PreCommit-SENTRY-Build%402/sentry-provider/sentry-provider-db/target/test-classes/keystore.jks
(No such file or directory)"
3) Some discussion online shows the slave node may use the wrong root
(top-level) directory.
https://issues.jenkins-ci.org/browse/JENKINS-17563
java.io.FileNotFoundException: Engineering/iOS/UScapeIt/UScapeIt.app.ipa (No
such file or directory)
"I believe the directory is fine.
The file reference should be relative to the workspace directory.
The job is running on a mac slave node.
Apparently, unlike the directory organization on the master node, it uses
workspace instead of jobs as a top-level directory.
Also, instead of having a workspace subdirectory, the directory matching the
jobname is the workspace."
> [Flaky test] TestSentryWebServerWithSSL times out
> -
>
> Key: SENTRY-1400
> URL: https://issues.apache.org/jira/browse/SENTRY-1400
> Project: Sentry
> Issue Type: Test
>Reporter: Sravya Tirukkovalur
>Assignee: Na Li
>
> I atleast saw two precommit jobs which failed due to this:
> https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console
> https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console
> [~lili] any idea whats going on?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (SENTRY-1400) [Flaky test] TestSentryWebServerWithSSL times out
[ https://issues.apache.org/jira/browse/SENTRY-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li reassigned SENTRY-1400: - Assignee: Na Li > [Flaky test] TestSentryWebServerWithSSL times out > - > > Key: SENTRY-1400 > URL: https://issues.apache.org/jira/browse/SENTRY-1400 > Project: Sentry > Issue Type: Test >Reporter: Sravya Tirukkovalur >Assignee: Na Li > > I atleast saw two precommit jobs which failed due to this: > https://builds.apache.org/job/PreCommit-SENTRY-Build/1872/console > https://builds.apache.org/job/PreCommit-SENTRY-Build/1874/console > [~lili] any idea whats going on? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (SENTRY-1723) Flaky test TestHDFSIntegrationEnd2End
[
https://issues.apache.org/jira/browse/SENTRY-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Na Li reassigned SENTRY-1723:
-
Assignee: Na Li
> Flaky test TestHDFSIntegrationEnd2End
> -
>
> Key: SENTRY-1723
> URL: https://issues.apache.org/jira/browse/SENTRY-1723
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Reporter: Alexander Kolbasov
>Assignee: Na Li
> Fix For: sentry-ha-redesign
>
>
> When I run the {{TestHDFSIntegrationEnd2End}} test from sentry-ha-redesign
> branch, I get
> {code}
> ---
> T E S T S
> ---
> Running org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 142.788 sec
> <<< FAILURE! - in org.apa
> che.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End
> testColumnPrivileges(org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End)
> Time elapsed: 15.0
> 14 sec <<< FAILURE!
> java.lang.AssertionError: Error at verifying Path action :
> /user/hive/warehouse/db2.db/p1 ; expected:
> but was:
> at
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End.testColumnPrivileges(TestHDFSI
> ntegrationEnd2End.java:505)
> Results :
> Flaked tests:
> org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End.testColumnPrivileges(org.apache.sentry.te
> sts.e2e.hdfs.TestHDFSIntegrationEnd2End)
> Run 1:
> TestHDFSIntegrationEnd2End.testColumnPrivileges:505->TestHDFSIntegrationBase.verifyOnAllSubDirs:186->TestHDFSIntegrationBase.verifyOnAllSubDirs:205->TestHDFSIntegrationBase.verifyOnAllSubDirs:209->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:234->TestHDFSIntegrationBase.verifyOnAllSubDirsHelper:224
> Error at verifying Path action : /user/hive/warehouse/db2.db/p1 ;
> expected: but was:
> Run 2: PASS
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Flakes: 1
> {code}
> I run the test with
> {code}
> mvn clean -pl sentry-tests/sentry-tests-hive -am -DfailIfNoTests=false
> -Dtest=TestHDFSIntegrationEnd2End test
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (SENTRY-1698) PathsUpdate.parsePath() calls FileSystem.getDefaultUri() way too often
[
https://issues.apache.org/jira/browse/SENTRY-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov resolved SENTRY-1698.
Resolution: Fixed
Fix Version/s: sentry-ha-redesign
Fixed as part of SENTRY-1687
> PathsUpdate.parsePath() calls FileSystem.getDefaultUri() way too often
> --
>
> Key: SENTRY-1698
> URL: https://issues.apache.org/jira/browse/SENTRY-1698
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Labels: bite-sized, newbie
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1698.001.patch
>
>
> The PathsUpdate#parsePath() method has the following code:
> {code}
> String scheme = uri.getScheme();
> if (scheme == null) {
> // Use the default URI scheme only if the path has no scheme.
> URI defaultUri = FileSystem.getDefaultUri(CONF);
> {code}
> The default URI never changes so there is no need to call
> FileSystem.getDefaultUri(CONF) for each input string which doesn't have URI.
> This should be done once in constructor.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1698) PathsUpdate.parsePath() calls FileSystem.getDefaultUri() way too often
[
https://issues.apache.org/jira/browse/SENTRY-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1698:
---
Issue Type: Sub-task (was: Bug)
Parent: SENTRY-872
> PathsUpdate.parsePath() calls FileSystem.getDefaultUri() way too often
> --
>
> Key: SENTRY-1698
> URL: https://issues.apache.org/jira/browse/SENTRY-1698
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: 1.8.0, sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Labels: bite-sized, newbie
> Attachments: SENTRY-1698.001.patch
>
>
> The PathsUpdate#parsePath() method has the following code:
> {code}
> String scheme = uri.getScheme();
> if (scheme == null) {
> // Use the default URI scheme only if the path has no scheme.
> URI defaultUri = FileSystem.getDefaultUri(CONF);
> {code}
> The default URI never changes so there is no need to call
> FileSystem.getDefaultUri(CONF) for each input string which doesn't have URI.
> This should be done once in constructor.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (SENTRY-1700) FullUpdateInitializer should not use preconditions to verify HMS data
[ https://issues.apache.org/jira/browse/SENTRY-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov resolved SENTRY-1700. Resolution: Fixed Addressed as part of SENTRY-1687 > FullUpdateInitializer should not use preconditions to verify HMS data > - > > Key: SENTRY-1700 > URL: https://issues.apache.org/jira/browse/SENTRY-1700 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Minor > > The FullUpdateInitializer class uses preconditions to verify data obtained > from HMS. The problem is that precondition failure is an unchecked exception > which will make HMSFollower unusable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (SENTRY-1606) Sentry HDFS Sync should survive in presence of bad paths objects
[ https://issues.apache.org/jira/browse/SENTRY-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov resolved SENTRY-1606. Resolution: Fixed Fix Version/s: sentry-ha-redesign Addressed as part of SENTRY-1687 > Sentry HDFS Sync should survive in presence of bad paths objects > - > > Key: SENTRY-1606 > URL: https://issues.apache.org/jira/browse/SENTRY-1606 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov > Fix For: sentry-ha-redesign > > > Currently when Sentry encounters any problem with the path object it fails to > construct a snapshot and the whole HDFS sync process breaks. > To make things more robust it should skip bad paths and continue creating the > snapshot. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1687:
---
Resolution: Fixed
Fix Version/s: sentry-ha-redesign
Status: Resolved (was: Patch Available)
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1687.001-sentry-ha-redesign.patch
>
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1649: -- Attachment: SENTRY-1649.031-sentry-ha-redesign.patch v31. rebased on latest commits. identical to v30 > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.015-sentry-ha-redesign.patch, > SENTRY-1649.022-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.025-sentry-ha-redesign.patch, > SENTRY-1649.028-sentry-ha-redesign.patch, > SENTRY-1649.029-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.031-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[ https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984924#comment-15984924 ] Na Li commented on SENTRY-1649: --- build 2573 and 2576 failed at "TestHDFSIntegrationEnd2End.testColumnPrivileges". It is not a stable test. And is reported in (SENTRY-1723) Flaky test TestHDFSIntegrationEnd2End. "Error Message Error at verifying Path action : /user/hive/warehouse/db2.db/p1 ; expected: but was: Stacktrace java.lang.AssertionError: Error at verifying Path action : /user/hive/warehouse/db2.db/p1 ; expected: but was: at org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End.testColumnPrivileges(TestHDFSIntegrationEnd2End.java:505)" > Initialize HMSFollower when sentry server actually starts > - > > Key: SENTRY-1649 > URL: https://issues.apache.org/jira/browse/SENTRY-1649 > Project: Sentry > Issue Type: Sub-task > Components: Hdfs Plugin >Affects Versions: sentry-ha-redesign >Reporter: Hao Hao >Assignee: Na Li >Priority: Critical > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1649.015-sentry-ha-redesign.patch, > SENTRY-1649.022-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.024-sentry-ha-redesign.patch, > SENTRY-1649.025-sentry-ha-redesign.patch, > SENTRY-1649.028-sentry-ha-redesign.patch, > SENTRY-1649.029-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch, > SENTRY-1649.030-sentry-ha-redesign.patch > > > Now HMSFollower has been initialized at the constructor of SentryService. It > would be better to initialize it when the service starts, e.g runServer(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[
https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984587#comment-15984587
]
Hadoop QA commented on SENTRY-1649:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865055/SENTRY-1649.030-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2576/console
This message is automatically generated.
> Initialize HMSFollower when sentry server actually starts
> -
>
> Key: SENTRY-1649
> URL: https://issues.apache.org/jira/browse/SENTRY-1649
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Na Li
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1649.015-sentry-ha-redesign.patch,
> SENTRY-1649.022-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.025-sentry-ha-redesign.patch,
> SENTRY-1649.028-sentry-ha-redesign.patch,
> SENTRY-1649.029-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch
>
>
> Now HMSFollower has been initialized at the constructor of SentryService. It
> would be better to initialize it when the service starts, e.g runServer().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1593) Implement client failover for Generic and NN clients
[
https://issues.apache.org/jira/browse/SENTRY-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984454#comment-15984454
]
Hadoop QA commented on SENTRY-1593:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865056/SENTRY-1593.010-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2575/console
This message is automatically generated.
> Implement client failover for Generic and NN clients
>
>
> Key: SENTRY-1593
> URL: https://issues.apache.org/jira/browse/SENTRY-1593
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: kalyan kumar kalvagadda
>Priority: Blocker
> Labels: HA
> Fix For: sentry-ha-redesign
>
> Attachments: old_service_client_class_diagram.png,
> SENTRY-1593.001-sentry-ha-redesign.patch,
> SENTRY-1593.002-sentry-ha-redesign.patch,
> SENTRY-1593.003-sentry-ha-redesign.patch,
> SENTRY-1593.004-sentry-ha-redesign.patch,
> SENTRY-1593.005-sentry-ha-redesign.patch,
> SENTRY-1593.006-sentry-ha-redesign.patch,
> SENTRY-1593.007-sentry-ha-redesign.patch,
> SENTRY-1593.009-sentry-ha-redesign.patch,
> SENTRY-1593.010-sentry-ha-redesign.patch,
> service_client_class_diagram_latest_v1.png, service_client_class_diagram.png
>
>
> We need to have client failover logic for Generic service clients and Name
> Node clients. Currently only db policy clients have it implemented.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984446#comment-15984446
]
Hadoop QA commented on SENTRY-1687:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865079/SENTRY-1687.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 3 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
{color:red}ERROR:{color} Failed:
org.apache.sentry.provider.db.service.persistent.TestSentryStore
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2574/console
This message is automatically generated.
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: SENTRY-1687.001-sentry-ha-redesign.patch
>
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1593) Implement client failover for Generic and NN clients
[
https://issues.apache.org/jira/browse/SENTRY-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984379#comment-15984379
]
Hadoop QA commented on SENTRY-1593:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865056/SENTRY-1593.010-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:green}Overall:{color} +1 all checks pass
{color:green}SUCCESS:{color} all tests passed
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2572/console
This message is automatically generated.
> Implement client failover for Generic and NN clients
>
>
> Key: SENTRY-1593
> URL: https://issues.apache.org/jira/browse/SENTRY-1593
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: kalyan kumar kalvagadda
>Priority: Blocker
> Labels: HA
> Fix For: sentry-ha-redesign
>
> Attachments: old_service_client_class_diagram.png,
> SENTRY-1593.001-sentry-ha-redesign.patch,
> SENTRY-1593.002-sentry-ha-redesign.patch,
> SENTRY-1593.003-sentry-ha-redesign.patch,
> SENTRY-1593.004-sentry-ha-redesign.patch,
> SENTRY-1593.005-sentry-ha-redesign.patch,
> SENTRY-1593.006-sentry-ha-redesign.patch,
> SENTRY-1593.007-sentry-ha-redesign.patch,
> SENTRY-1593.009-sentry-ha-redesign.patch,
> SENTRY-1593.010-sentry-ha-redesign.patch,
> service_client_class_diagram_latest_v1.png, service_client_class_diagram.png
>
>
> We need to have client failover logic for Generic service clients and Name
> Node clients. Currently only db policy clients have it implemented.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (SENTRY-1649) Initialize HMSFollower when sentry server actually starts
[
https://issues.apache.org/jira/browse/SENTRY-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984374#comment-15984374
]
Hadoop QA commented on SENTRY-1649:
---
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12865055/SENTRY-1649.030-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2573/console
This message is automatically generated.
> Initialize HMSFollower when sentry server actually starts
> -
>
> Key: SENTRY-1649
> URL: https://issues.apache.org/jira/browse/SENTRY-1649
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
>Affects Versions: sentry-ha-redesign
>Reporter: Hao Hao
>Assignee: Na Li
>Priority: Critical
> Fix For: sentry-ha-redesign
>
> Attachments: SENTRY-1649.015-sentry-ha-redesign.patch,
> SENTRY-1649.022-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.024-sentry-ha-redesign.patch,
> SENTRY-1649.025-sentry-ha-redesign.patch,
> SENTRY-1649.028-sentry-ha-redesign.patch,
> SENTRY-1649.029-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch,
> SENTRY-1649.030-sentry-ha-redesign.patch
>
>
> Now HMSFollower has been initialized at the constructor of SentryService. It
> would be better to initialize it when the service starts, e.g runServer().
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (SENTRY-1725) Include response status in TSentrySyncIDResponse
Alexander Kolbasov created SENTRY-1725:
--
Summary: Include response status in TSentrySyncIDResponse
Key: SENTRY-1725
URL: https://issues.apache.org/jira/browse/SENTRY-1725
Project: Sentry
Issue Type: Sub-task
Components: Sentry
Affects Versions: sentry-ha-redesign
Reporter: Alexander Kolbasov
Assignee: Alexander Kolbasov
All responses for the policy model include
{{sentry_common_service.TSentryResponseStatus}}, but {{TSentrySyncIDResponse}}
doesn't have it (by mistake). We should add it to be consistent with other
responses.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1685) Port SENTRY-1489 to sentry-ha-redesign branch
[ https://issues.apache.org/jira/browse/SENTRY-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated SENTRY-1685: --- Resolution: Fixed Status: Resolved (was: Patch Available) [~lina...@cloudera.com] Thank you fr your contribution! > Port SENTRY-1489 to sentry-ha-redesign branch > - > > Key: SENTRY-1685 > URL: https://issues.apache.org/jira/browse/SENTRY-1685 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Na Li > Labels: merge > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1685.001-sentry-ha-redesign.patch, > SENTRY-1685.002-sentry-ha-redesign.patch > > > We need to port SENTRY-1489 to sentry-ha-redesign branch -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (SENTRY-1685) Port SENTRY-1489 to sentry-ha-redesign branch
[ https://issues.apache.org/jira/browse/SENTRY-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984285#comment-15984285 ] Alexander Kolbasov edited comment on SENTRY-1685 at 4/26/17 7:10 AM: - [~lina...@cloudera.com] Thank you for your contribution! The patch is submitted with the original commit message. was (Author: akolb): [~lina...@cloudera.com] Thank you fr your contribution! > Port SENTRY-1489 to sentry-ha-redesign branch > - > > Key: SENTRY-1685 > URL: https://issues.apache.org/jira/browse/SENTRY-1685 > Project: Sentry > Issue Type: Sub-task > Components: Sentry >Affects Versions: sentry-ha-redesign >Reporter: Alexander Kolbasov >Assignee: Na Li > Labels: merge > Fix For: sentry-ha-redesign > > Attachments: SENTRY-1685.001-sentry-ha-redesign.patch, > SENTRY-1685.002-sentry-ha-redesign.patch > > > We need to port SENTRY-1489 to sentry-ha-redesign branch -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1489) Categorize e2e tests into slow and regular tests, so that can adapt the timeout and etc.
[ https://issues.apache.org/jira/browse/SENTRY-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated SENTRY-1489: --- Fix Version/s: sentry-ha-redesign > Categorize e2e tests into slow and regular tests, so that can adapt the > timeout and etc. > > > Key: SENTRY-1489 > URL: https://issues.apache.org/jira/browse/SENTRY-1489 > Project: Sentry > Issue Type: Test > Components: Sentry >Affects Versions: 1.8.0, sentry-ha-redesign >Reporter: Anne Yu >Assignee: Anne Yu > Fix For: 1.8.0, sentry-ha-redesign > > Attachments: SENTRY-1489.0.patch, SENTRY-1489.1.patch, > SENTRY-1489.2.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1687:
---
Status: Patch Available (was: Open)
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: SENTRY-1687.001-sentry-ha-redesign.patch
>
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1687:
---
Status: Open (was: Patch Available)
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1687:
---
Attachment: SENTRY-1687.001-sentry-ha-redesign.patch
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
> Attachments: SENTRY-1687.001-sentry-ha-redesign.patch
>
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (SENTRY-1687) FullUpdateInitializer can be more efficient
[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated SENTRY-1687:
---
Attachment: (was: SENTRY-1687.001-sentry-ha-redesign.patch)
> FullUpdateInitializer can be more efficient
> ---
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
>Affects Versions: sentry-ha-redesign
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
