Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 26, 2015, 8:53 p.m.)


Review request for hive.


Changes
---

added doDbSpecificInitializationsBeforeQuery in runTestQuery


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)

https://issues.apache.org/jira/browse/HIVE-9741


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 25, 2015, 11:58 p.m.)


Review request for hive.


Changes
---

remved setDbANSIMode


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)

https://issues.apache.org/jira/browse/HIVE-9741


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 25, 2015, 9:37 p.m.)


Review request for hive.


Changes
---

1. moved setDbANSIMode call
2. removed logging in comment


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)

https://issues.apache.org/jira/browse/HIVE-9741


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 25, 2015, 9:24 p.m.)


Review request for hive.


Repository: hive-git


Description (updated)
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)

https://issues.apache.org/jira/browse/HIVE-9741


Diffs
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/#review73950
---



metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java


this logging statement is invalid now it appears


- Sergey Shelukhin


On Feb. 24, 2015, 10:34 p.m., XIAOBING ZHOU wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31249/
> ---
> 
> (Updated Feb. 24, 2015, 10:34 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> MetaStoreDirectSql constructor is querying DB to determine dbType, which 
> leads to too many DB queries to make megastore slow as ObjectStore.setConf 
> might be called frequently. Moreover, ObjectStore.setConf begins/ends with 
> lock acquire/release, if the underlying DB hangs somehow, lock is never 
> released and all hereafter incoming requests are blocked.
> 
> Two points:
> 1. Using getProductName based JDBC driver to get dbType info.
> 2. Since metastore auto-creaton is disabled by default, it'd better bypass 
> ensureDbInit() and runTestQuery() in order to avoid DB queries within 
> critical section of setConf.
> 
> Here’s stack trace:
> MetaStoreDirectSql.determineDbType(...)
> MetaStoreDirectSql.MetaStoreDirectSql(...)
> ObjectStore.initialize(...)
> ObjectStore.setConf(…)
> 
> 
> Diffs
> -
> 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
> 574141c 
> 
> Diff: https://reviews.apache.org/r/31249/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> XIAOBING ZHOU
> 
>

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 24, 2015, 10:34 p.m.)


Review request for hive.


Changes
---

removed empty spaces.


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 24, 2015, 10:24 p.m.)


Review request for hive.


Changes
---

moved setDbANSIMode call to MetaStoreDirectSql


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)


Diffs (updated)
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU

Re: Review Request 31249: Refactor MetaStoreDirectSql constructor by removing DB queries out of critical section

[XIAOBING ZHOU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31249/
---

(Updated Feb. 24, 2015, 12:49 a.m.)


Review request for hive.


Summary (updated)
-

Refactor MetaStoreDirectSql constructor by removing DB queries out of critical 
section


Repository: hive-git


Description
---

MetaStoreDirectSql constructor is querying DB to determine dbType, which leads 
to too many DB queries to make megastore slow as ObjectStore.setConf might be 
called frequently. Moreover, ObjectStore.setConf begins/ends with lock 
acquire/release, if the underlying DB hangs somehow, lock is never released and 
all hereafter incoming requests are blocked.

Two points:
1. Using getProductName based JDBC driver to get dbType info.
2. Since metastore auto-creaton is disabled by default, it'd better bypass 
ensureDbInit() and runTestQuery() in order to avoid DB queries within critical 
section of setConf.

Here’s stack trace:
MetaStoreDirectSql.determineDbType(...)
MetaStoreDirectSql.MetaStoreDirectSql(...)
ObjectStore.initialize(...)
ObjectStore.setConf(…)


Diffs
-

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
574141c 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java fcaffc7 

Diff: https://reviews.apache.org/r/31249/diff/


Testing
---


Thanks,

XIAOBING ZHOU