Matthias, I would like to move our discussion to oscar-devel so that many people can see our discussion and participate in it if they want.
Matthias Blankenhaus wrote: > On Fri, 19 Jan 2007, DongInn Kim wrote: > > >> Well, before we go further, can we please talk about why this >> integration is really necessary and that this matters with only ganglia? >> > > I have worked for several years on Enterprise Mgmt Sofware. As we added > more and more software modules it became clear that we needed one source > for the cluster model to avoid a skew of the model across various co- > operating modules. This could lead into catastrophic scenarios. > Simply, dublicate model have the potential for a temporary window > of inconsistency among co-operating SW modules that depent on an > identical view of the system. > Eventually, we ended with one centralized view of the cluster and a > system-wide information bus providing a client API for the system model. > The system bus greatly simplified the communication among co-operating > modules by providing a single, proofen methods for exchaning > information via message passing. > > Yes, I understand what you are pursuing on your cluster but I am not still convinced of your idea that ODA needs to be integrated with the data(configuration, installation info, and running data/log (?) ) of each package because OSCAR packages are fully and consistently configured and all the necessary data/configuration are saved in ODA. For example, in ganglia, do we have anything else to add/modify in ODA? As you may know, OSCAR has defined/saved in config.xml the specific configuration of each package which you call a module in your sentence. The configuration file (config.xml) of each package includes the basic information to install/configure the package. If necessary, all the installation OSCAR API of each package enable to configure all the specific setup for the package during the OSCAR installation. So, OSCAR can provide the consistent package managing scheme for the users. As long as you set up your cluster with OSCAR packaging scheme, your cluster would not be in trouble that you mentioned above. If you are not fully comfortable with my expression of OSCAR package API and config.xml, please refer to the following link. http://oscar.openclustergroup.org/public/docs/devel/oscarpkg-howto_22jan04.pdf >> I am not saying that ganglia does not need to use OSCAR >> database but I am saying that if necessary ganglia can simply queries >> via a perl script to use ODA. >> > > As I learned over the pass of the last days, Ganglia and OSCAR and C3 are > indeed synchronized in terms of the cluster model. It appears that > during cluster definition via OSCAR Wizard various scripts generate > the Ganglia / C3 cluster configuration files. > Again, while this is good news and presents a good starting point, I do > not believe that this will be sufficient in the long run, as discussed > above. > > >> Even so, do you think that the integration is easy to setup and to maintain? >> > > Not that easy. Firstly, we would need to extend ODA to accommodate for > Ganglia and C3 and any other module that depends on the cluster > configuration. Starting from a certain ODA access rate we would need to > think about caching algorithms. Secondly, we would have to devise a > generic information bus. Of course, this has been done before and the > requirements for this > are understood for the most part. Thirdly, we would have to convince > the OSCAR communitity to except this model and add a client API exported > via the information bus to the package API. > As I mentioned before, ODA is only used for OSCAR installation and once a cluster is installed/configured, there is really nothing to do with ODA now. I have never worried about database access problems during the OSCAR installation because OSCAR has a single database access and we don't need to have multiple threads to get several database accesses at the same time. Are you saying that we still need to add/modify some information of packages (modules) to ODA during installation? If so, can you please let me know what they are? Or if you are saying that we need to integrate the data while cluster is running after the OSCAR installation, what kind of information should be kept in ODA and integrated? And I doubt that we need to keep this run-time information in ODA. > >> So, what is the goal of the integration? >> > I believe I have answered that question. > > >> BTW, does ganglia use its own database? >> > Yes it does. > What kind of database does it use? Is it one of the normal databases(mysql, pgsql, sqlite, and so on) that most people use or kind of database with a particular database format? I could not find any database files or data on my OSCAR cluster. Regards, - DongInn >> >> >> Matthias Blankenhaus wrote: >> >>> On Fri, 19 Jan 2007, DongInn Kim wrote: >>> >>> >>> >>>> Hi Matthias, >>>> >>>> >>>> >>>>> Is there an integration point between ODA, Ganglia and C3 ? >>>>> >>>>> >>>> ODA has been used only to keep track of the OSCAR installation process. >>>> Currently it is more focusing on package management and I think the >>>> present scheme is pretty ready for this but I am not sure for further. >>>> Can you tell me what integration you are talking about between >>>> Packages(i.e., Ganglia, C3, PVM, and so on) and ODA? >>>> >>>> >>> Looking at ODA it holds some fundamental structural information about the >>> cluster: it contains the nodes contained within a cluster. These >>> nodes essentially constitue a specific cluster. >>> Ganglia also keeps a list of nodes belong a specific cluster (or even >>> sub-cluster). >>> So both pieces keep a model of the cluster in some form or the other. >>> If these models are completely separate then there is the concern of >>> a temp. different view on the system, depending on how the model in >>> each system is maintained. Thus, in order to keep a consistent view >>> on the cluster it would be necessary for both systems to reference the >>> exact same model. For instance, they could both use the ODA (MySQL DB) >>> from OSCAR. >>> In summary, OSCAR could provide a centralized object model for the >>> cluster used by all components that are based on it. Naturally, OSCAR >>> would provide access to this central cluster view by a client API. >>> This API can be done in several ways depending on what this model is >>> intentend for. I imagine at least two model here: synchronous and >>> asynchronous. Access to this central repository would need to have >>> transactional semantics following the ACID model. Also, caching >>> should be in place for performance reasons. >>> >>> I hope I was to express my thoughts clearly. If not, then please work >>> with me to get my point across. >>> >>> The other questions than would now be in a different context then you >>> answered them before. >>> >>> Thanx again, >>> Matthias >>> >>> >>> >>> >>> >>>> Regards, >>>> >>>> >>>> Matthias Blankenhaus wrote: >>>> >>>> >>>>> DingInn, >>>>> >>>>> thank you vey much for the docu. This is better than anything else >>>>> I have found so far. I already saw some interesting info about ODA. >>>>> >>>>> Let me ask you another question that you may know the answer for: >>>>> Is there an integration point between ODA, Ganglia and C3 ? Do >>>>> they feed of the same cluster model (=DB) or do they live in a >>>>> parallel universe ? If so, did anyone ever attempt to consolidate >>>>> the different views ? >>>>> >>>>> >>>>> >>>>> >>>>>> So, please let us know if you need any further information. BTW, it is >>>>>> always welcome that you can devote your some ideas/experiences to oscar >>>>>> if you have something to share. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> Yes indeed, we will contribute to OSCAR. We most likely need to change >>>>> the DB tables as well as we intend to employ a 3-tier HPC architecture. >>>>> It would be my pleasure to coordinate with you on that. >>>>> >>>>> Regards, >>>>> Matthias >>>>> >>>>> >>>>> >>>>> >>>>>> Regards, >>>>>> >>>>>> - DongInn >>>>>> >>>>>> Matthias Blankenhaus wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On Thu, 18 Jan 2007, DongInn Kim wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Hi Matthias, >>>>>>>> >>>>>>>> Yes, I have two papers about ODA but I think they were written with a >>>>>>>> little old database structure and you may need to look at >>>>>>>> oda/scripts/oscar_table.sql to figure out the current scheme. >>>>>>>> http://svn.oscar.openclustergroup.org/trac/oscar/browser/trunk/packages/oda/scripts/oscar_table.sql >>>>>>>> >>>>>>>> My papers can be found at the following link. >>>>>>>> http://www.osl.iu.edu/publications/Author/KIM-DI.php >>>>>>>> >>>>>>>> I hope that these are what you are looking for. >>>>>>>> Please let me know if you need anything else. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Thanx DongInn, this already helps a lot. >>>>>>> I have got one question: Do you happen to know if there is a over >>>>>>> OSCAR >>>>>>> design document. I was not able to find it on the Web. I am trying to >>>>>>> figure out how the various components tie together. >>>>>>> >>>>>>> Thanx, >>>>>>> Matthias >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> - DongInn >>>>>>>> >>>>>>>> Matthias Blankenhaus wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Hello ! >>>>>>>>> >>>>>>>>> I am currently working on a HPC project involving OSCAR. I need to >>>>>>>>> learn >>>>>>>>> about OSCAR's architecture and came across your paper with the title >>>>>>>>> " The Introduction of the OSCAR Database API (ODA)". >>>>>>>>> >>>>>>>>> I was wondering if you could send me an electronic copy of that paper >>>>>>>>> ? >>>>>>>>> Do you happen to have more paper or links to paper about OSCAR's >>>>>>>>> internal >>>>>>>>> architecture ? >>>>>>>>> >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> >>>>>>>>> Matthias >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Oscar-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/oscar-devel
