Hi, We are trying to make a UI for our HBase + Hadoop applications.
Basically, what we want is a web front end that can be hosted on a different server and present data from HBase, as well as allow us to launch Map-Reduce jobs from the web browser. Our current approach is as follows: We have a 6-node development cluster with each node running Scientific Linux in a VM with the following configuration: Node - Hostname - Daemons -------------------------------------------- Node 1 - "namenode" - namenode, secondarynamenode, regionserver, hbase-master, zookeeper Node 2 - "jobtracker" - jobtracker, zookeeper Node 3 - "slave0" - zookeeper, datanode, tasktracker Node 4 - "slave1" - datanode, tasktracker Node 5 - "slave2" - datanode, tasktracker Node 6 - "slave3" - datanode, tasktracker We have a 7th scientific linux VM running a Grails web application called "BillyWeb". We have created several Map-Reduce applications that run on the namenode to process HBase data and populate a results table. Currently, these MR apps are run from the command line on namenode. We use the HBase REST interface to query data from the results table and present it in an AJAX-enabled web page. That is the reading part of the problem solved :) Now we need to have a HTML page with a button, that when clicked will execute an MR app on the namenode. This forms the writing part of the problem. Currently, we are attempting to do this using Java's Remote Method Invocation (RMI) feature. I have successfully created a Java client-server application where the server program runs on "namenode" and the client runs on "BillyWeb" successfully. The client program calls a remote object method to trigger the MR job on namenode and gets the result. We are now in the process of integrating it into the Grails webapp, by adding the source code and calling it using groovy taglibs. The only remaining issue we are wrestling with is the need to modify the security policy of the web app to allow access to the client portions of the source code to the remote server. Eventually we hope to have an internal web site that you can visit in any web browser to view custom-made visualisations of the data and execute parameterised MR jobs to process data stored in HBase (amongst other places). We would like to know your thoughts on this approach. In particular, this feels like a potentially convoluted approach to building a front-end that may feature several redundant steps (are we are reinventing too many wheels?). Is there an alternative approach that you can think of that might be more sensible? Can you think of any problems with the approach we are currently taking? Thanks, Tom
