-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22997/
-----------------------------------------------------------
Review request for Tajo.
Bugs: TAJO-611
https://issues.apache.org/jira/browse/TAJO-611
Repository: tajo
Description
-------
As we talked offline, high availability is one of our next goal. Service
discovery can help us maintain health statuses for all daemons ( master and
workers) . Meanwhile, those daemons can find each other easily. Furthermore,
it's very useful for my current work TAJO-540 cuz it can randomly select nodes
for tajo scheduler.
One of the best candidates is Netflix Curator.
http://curator.apache.org/curator-x-discovery/index.html
I'd like to introduce Xuhui to help us with this issue. Xuhui was my colleague
at Alibaba group. He was active in Hive community, one of his job is adding
multi-distinct aggregation feature to Hive. Currently, he is a researcher work
at Microsoft.
Mafish
Below is comment on this issue from Xuhui
To my understanding, this feature is for high availability as well as high
scalability. We don't need to provide all machine info for every service when
tajo starts. Instead, we can dynamically register services to service discovery
if necessary. Also, if some machine fails, it can be easily detected and
replaced.
Diffs
-----
tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java 96a2691
tajo-core/pom.xml 32ae266
tajo-core/src/main/java/org/apache/tajo/curator/CuratorConstants.java
PRE-CREATION
tajo-core/src/main/java/org/apache/tajo/curator/ServiceInstanceSerializer.java
PRE-CREATION
tajo-core/src/main/java/org/apache/tajo/curator/zk/ZKCurator.java
PRE-CREATION
tajo-core/src/main/proto/CuratorProtocol.proto PRE-CREATION
tajo-core/src/test/java/org/apache/tajo/curator/zk/TestZKCurator.java
PRE-CREATION
Diff: https://reviews.apache.org/r/22997/diff/
Testing
-------
mvn clean install
Thanks,
Jung JaeHwa
