Re: [VOTE] Move OODT to Attic?

[Rajith Siriwardana]

+1 move to the attic.

/Rajith

On Wed, Apr 5, 2023 at 8:08 AM Lewis John McGibbney 
wrote:

> +1 move to the attic.
> I share Sean's sentiment entirely. A real success story.
> Thanks Imesha for representing the project to the Board.
> lewismc
>
> On 2023/04/03 01:02:01 Imesha Sudasingha wrote:
> > Hello everyone:
> >
> > Due to inactivity, Apache OODT is considering moving to the Attic [1].
> This
> > email serves as a call to all PMC members to vote whether to retire OODT
> to
> > the attic, or not.  Note that three -1 votes will be sufficient to cancel
> > retirement to the attic no matter how many +1 votes there are.
> >
> > PMC members, please reply to this email with your vote:
> >
> > +1 [ ] I wish for Apache OODT to be retired to the Apache Attic
> > +0 [ ] I do not care
> > -1 [ ] Apache OODT should not be retired to the Attic
> >
> > Here's my +1.
> >
> > Thanks,
> > Imesha
> >
> > [1] https://attic.apache.org/
> >
>

Re: Congratulations on your new role at Apache

[Rajith Siriwardana]

Congratulations Imesha!

Cheers,
Rajith

On Thu, Apr 19, 2018 at 8:50 AM, Imesha Sudasingha 
wrote:

> Thanks Tom! :D
>
> - Imesha
>
>
>
>
> On 19 April 2018 at 01:00, Tom Barber  wrote:
>
> > Woop!
> >
> > Congrats to our new Chair, Imesha.
> >
> >
> >
> >
> > On 18/04/18 20:21, Craig L Russell wrote:
> >
> >> Dear new PMC chairs,
> >>
> >> Congratulations on your new role at Apache. I've changed your LDAP
> >> privileges to reflect your new status.
> >>
> >> Please read this and update the foundation records:
> >> https://svn.apache.org/repos/private/foundation/officers/adv
> >> ice-for-new-pmc-chairs.txt
> >>
> >> Warm regards,
> >>
> >> Craig L Russell
> >>
> >
> >
> >
> > --
> >
> >
> > Spicule Limited is registered in England & Wales. Company Number:
> > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston
> > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> >
> >
> >
> >
> > All engagements are subject to Spicule Terms and Conditions of Business.
> > This email and its contents are intended solely for the individual to
> whom
> > it is addressed and may contain information that is confidential,
> > privileged or otherwise protected from disclosure, distributing or
> copying.
> > Any views or opinions presented in this email are solely those of the
> > author and do not necessarily represent those of Spicule Limited. The
> > company accepts no liability for any damage caused by any virus
> transmitted
> > by this email. If you have received this message in error, please notify
> us
> > immediately by reply email before deleting it from your system. Service
> of
> > legal notice cannot be effected on Spicule Limited by email.
> >
>

Re: Welcome Imesha

[Rajith Siriwardana]

Imesha welcome aboard!

On Tue, Aug 29, 2017 at 1:21 AM, Chris Mattmann  wrote:

> Welcome Imesha!
>
>
>
> On 8/28/17, 7:56 AM, "Tom Barber"  wrote:
>
> Hi folks,
>
> For those who haven't been following, Imesha has been doing some great
> work
> in bringing Zookeeper config management to OODT, hopefully this is an
> ongoing thing and we can roll it out across the board to better aid
> distribution.
>
> That aside, as  ever we like to thank those who contribute and as such,
> Imesha is now a fully fledged OODT committer and PMC member.
>
> Congrats and welcome to Imesha.
>
> Tom
>
>
>
>

Re: Welcome Imesha

[Rajith Siriwardana]

Welcome Imesha

On Fri, May 5, 2017 at 7:20 AM, Imesha Sudasingha 
wrote:

> Hi Tom and all,
>
> Thanks for the feedbacks and support given so far to come up with a good
> proposal and get selected for GSoC 2017. I'm looking forward to start
> working on my project soon and do my best to implement distributed
> configuration management to OODT. Hope you all will guide me when possible
> throughout the project.
>
> Thank you!
>
> Kind Regards,
> *Imesha Sudasingha*
> Undergraduate of Department of Computer Science and  Engineering,
> University of Moratuwa,
> Sri Lanka.
>
>   
> 
> 
>
> On 4 May 2017 at 23:30, Tom Barber  wrote:
>
> > I'd like to say an official Hi to Imesha who is our 2017 GSOC student for
> > OODT.
> >
> > Imesha will be working on adding Zookeeper support to the OODT
> > configuration codebase.
> >
> >
> > Tom
> >
>

Re: New Committer: Dana Freeborn

[Rajith Siriwardana]

Welcome Dana!!



On Tue, Mar 31, 2015 at 11:58 AM, BW w...@apache.org wrote:

 Dana welcome aboard!

 -Bill

 On Tue, Mar 31, 2015 at 10:10 AM, Freeborn, Dana J (398G) 
 dana.j.freeb...@jpl.nasa.gov wrote:

  Thanks Chris.  I have subscribed to the dev email list and I'm happy to
  introduce myself.
 
  My name is Dana Freeborn and I am the Group Supervisor for the Science
  Data Management Systems group of the Instrument Software and Science Data
  Systems Section at JPL.  I have been involved with OODT for over a decade
  with a focus on its application as the framework for for the ingestion,
  management and application of airborne and space borne remote sensing
  data.  With the new challenges facing our upcoming NASA earth orbiting
  missions in this era of big data, I am looking forward to being a part of
  the community of thinkers and doers who will provide the solutions.
 
  My thanks to the PMC for inviting me to join the Apache OODT community.
  I'm very honored.
 
  Dana
 
  From: Chris Mattmann mattm...@apache.orgmailto:mattm...@apache.org
  Date: Saturday, March 28, 2015 1:21 PM
  To: dev@oodt.apache.orgmailto:dev@oodt.apache.org 
 dev@oodt.apache.org
  mailto:dev@oodt.apache.org
  Cc: cpi dana.j.freeb...@jpl.nasa.govmailto:
 dana.j.freeb...@jpl.nasa.gov
  
  Subject: Re: New Committer: Dana Freeborn
 
  Hi Dana,
 
  It would be great if you could introduce yourself since
  there are more folks here than JPL and NASA. Also not sure
  if you are subscribed to the dev@oodt.apache.orgmailto:
  dev@oodt.apache.org list yet.
  If you haven’t subscribed yet, from your preferred email
  send a blank email to dev-subscr...@oodt.apache.orgmailto:
  dev-subscr...@oodt.apache.org and please
  follow the instructions from there.
 
  Cheers,
  Chris
 
 
 
  -Original Message-
  From: Tom Barber tom.bar...@meteorite.bimailto:tom.bar...@meteorite.bi
 
  Reply-To: dev@oodt.apache.orgmailto:dev@oodt.apache.org
  Date: Saturday, March 28, 2015 at 3:13 AM
  To: dev@oodt.apache.orgmailto:dev@oodt.apache.org
  Subject: Re: New Committer: Dana Freeborn
 
  Hopefully I did it right! ;)
 
  On 28/03/15 00:40, Cameron Goodale wrote:
  That's excellent news Tom. Thanks for keeping the process moving
 
  Cameron
 
  On Friday, March 27, 2015, Tom Barber tom.bar...@meteorite.bimailto:
  tom.bar...@meteorite.bi wrote:
 
  The Project Management Committee (PMC) for Apache OODT has asked Dana
  Freeborn to become a committer and we are pleased to announce that they
  have accepted.
 
 
  -- Tom
 
 
 
 
  --
  *Tom Barber* | Technical Director
 
  meteorite bi
  *T:* +44 20 8133 3730
  *W:* www.meteorite.bi | *Skype:* meteorite.consulting
  *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG,
  UK
 
 
 
 

Re: [ANNOUNCE] Tyler Palsulich is now on the PMC too!

[Rajith Siriwardana]

Welcome Tyler..!!!


Cheers,
--Rajith


On Wed, Jul 23, 2014 at 1:06 AM, Michael Starch starc...@umich.edu wrote:

 Great!  Welcome, Tyler.

 As another new PMC I aggree that we should discuss a roadmap.

 -Michael
 On Jul 22, 2014 11:58 AM, Michael Joyce jo...@apache.org wrote:

 Yay! Welcome Tyler!!!


 -- Joyce


 On Tue, Jul 22, 2014 at 11:36 AM, Tom Barber tom.bar...@meteorite.bi
 wrote:

  Aye I'll +1 that, and move this over to dev@

 We have an ever expanding PMC, so lets do some Project Managing! ;)

 I think we need to come up with some roadmap goals, both regarding
 features, and also the existing code base, build systems and delivery.

 Personally I think it would be benificial to get a bunch of us on the
 phone and discuss it in real time (I'm GMT based which does make it a
 little tricker) rather than hash it out over a 6 month chain of emails.
 Like I said the other day, I don't really know how other projects manage
 this type of thing so those with more experience may want to chime in here.

 Regards

 Tom



 On 22/07/14 18:33, Lewis John Mcgibbney wrote:

   Hi Tyler,
  Please see Tom's thread from before the weekend.
  I say we have a full on discussion on establishing a roadmap.
  For us to establish and agree on this will be extremely beneficial for
 the project.
  Lewis


 On Tue, Jul 22, 2014 at 10:29 AM, Tyler Palsulich tpalsul...@gmail.com
 wrote:

 Hi All,

  Thanks for inviting me to join as a committer and PMC member! I'm
 happy to be part of the project. Any tips on where to dive in?

  Tyler

Re: Review Request 18944: [OODT-669] Disabling resource monitor in Assignment monitor feature

[Rajith Siriwardana]

 On May 2, 2014, 9:20 p.m., Chris Mattmann wrote:
  Rajith, I'm not sure I get this patch. Why do we have to disable the 
  Resource monitor for this to work? We used to have an AssignmentMonitor 
  that didn't need Ganglia up and running -- so we could simply set 
  AssignmentMonitor as the default Monitor and we were good. What's changed?

Chris, earlier it was just generating bogus values in the AssignmentMonitor. 
but now it has the ability to use a dynamically asigned resource monitor 
(currenlty we have only Ganglia), according to [1] disabling Ganglia should 
happend at that level I guess. AssignmentMonitor contains things like governing 
the resource assignments like features so I guess it's a higher level, dynamic 
assignment of resource monitor is a level below that. So AssignmentMonitor to 
work there should be a ResourceMonitor assigned to it. Correct me if I'm wrong.

[1] 
https://cwiki.apache.org/confluence/display/OODT/Ganglia+plugin+for+CAS+Resource+Manager


- Rajith


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18944/#review42060
---


On March 9, 2014, 4:44 a.m., Rajith Siriwardana wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/18944/
 ---
 
 (Updated March 9, 2014, 4:44 a.m.)
 
 
 Review request for oodt and Chris Mattmann.
 
 
 Bugs: OODT-669
 https://issues.apache.org/jira/browse/OODT-669
 
 
 Repository: oodt
 
 
 Description
 ---
 
 Patch contains disabling Resource Monitor feature in Assignment Monitor and 
 also some re-factored code for assignment monitor 
 
 
 Diffs
 -
 
   
 trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/AssignmentMonitor.java
  1575649 
   
 trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/disabledmonitor/DisabledResourceMonitor.java
  PRE-CREATION 
   
 trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/disabledmonitor/DisabledResourceMonitorFactory.java
  PRE-CREATION 
   
 trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaAdapter.java
  1575649 
   
 trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaResourceMonitor.java
  1575649 
   trunk/resource/src/main/resources/resource.properties 1575649 
   
 trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestAssignmentMonWithNoResourceMon.java
  PRE-CREATION 
   trunk/resource/src/testdata/test.disabledmon.resource.properties 
 PRE-CREATION 
 
 Diff: https://reviews.apache.org/r/18944/diff/
 
 
 Testing
 ---
 
 Disabled monitor with Assignment monitor was tested
 
 
 Thanks,
 
 Rajith Siriwardana
 

Review Request 18944: [OODT-669] Disabling resource monitor in Assignment monitor feature

[Rajith Siriwardana]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18944/
---

Review request for oodt and Chris Mattmann.


Bugs: OODT-669
https://issues.apache.org/jira/browse/OODT-669


Repository: oodt


Description
---

Patch contains disabling Resource Monitor feature in Assignment Monitor and 
also some re-factored code for assignment monitor 


Diffs
-

  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/AssignmentMonitor.java
 1575649 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/disabledmonitor/DisabledResourceMonitor.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/disabledmonitor/DisabledResourceMonitorFactory.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaAdapter.java
 1575649 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaResourceMonitor.java
 1575649 
  trunk/resource/src/main/resources/resource.properties 1575649 
  
trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestAssignmentMonWithNoResourceMon.java
 PRE-CREATION 
  trunk/resource/src/testdata/test.disabledmon.resource.properties PRE-CREATION 

Diff: https://reviews.apache.org/r/18944/diff/


Testing
---

Disabled monitor with Assignment monitor was tested


Thanks,

Rajith Siriwardana

Re: Resource Manager 0.7-SNAPSHOT and Ganglia Monitor

[Rajith Siriwardana]

hi Cameron,

AFAIK there is no way to disable the Ganglia plugin, so you'll have to
setup and run a Ganglia metadata daemon (Gmetad). I'll check that again.

Cheers,
Rajith


On Tue, Dec 10, 2013 at 2:53 AM, Cameron Goodale sigep...@gmail.com wrote:

 Hey Rajith and Devs,

 I am upgrading my resource manager 0.3 to the latest version (currently
 0.7-SNAPSHOT) and I have run into the Ganglia Monitor Plugin with the
 following errors:

 SEVERE: The required nodeId is not available: sneffels-11 :null
 java.lang.NullPointerException
 at

 org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
 at

 org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
 at java.lang.Thread.run(Thread.java:662)
 Dec 9, 2013 1:17:05 PM
 org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor getLoad
 SEVERE: The required nodeId is not available: kush-11 :null
 java.lang.NullPointerException
 at

 org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
 at

 org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
 at

 org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
 at java.lang.Thread.run(Thread.java:662)


 I get this error message for all 44 of the nodes I have defined in my
 nodes.xml config.


 A couple questions:

 1.  Can I simply run resource manager without the Plugin via a change in
 config?  I tried to comment out the config lines, but it didn't work.

 2.  If I cannot simply disconnect the Ganglia Dependency, then do I have to
 setup and run a Ganglia metadata daemon?


 I look forward to your reply,


 Cameron

 --

 Sent from a Tin Can attached to a String

Re: Resource Manager 0.7-SNAPSHOT and Ganglia Monitor

[Rajith Siriwardana]

Sure Chris, I'll fix that asap

Cheers,
Rajith


On Tue, Dec 10, 2013 at 9:11 AM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Dear Rajth we need a way to disable it it should be optional

 Sent from my iPhone

  On Dec 9, 2013, at 6:57 PM, Rajith Siriwardana 
 rajithsiriward...@gmail.com wrote:
 
  hi Cameron,
 
  AFAIK there is no way to disable the Ganglia plugin, so you'll have to
  setup and run a Ganglia metadata daemon (Gmetad). I'll check that again.
 
  Cheers,
  Rajith
 
 
  On Tue, Dec 10, 2013 at 2:53 AM, Cameron Goodale sigep...@gmail.com
 wrote:
 
  Hey Rajith and Devs,
 
  I am upgrading my resource manager 0.3 to the latest version (currently
  0.7-SNAPSHOT) and I have run into the Ganglia Monitor Plugin with the
  following errors:
 
  SEVERE: The required nodeId is not available: sneffels-11 :null
  java.lang.NullPointerException
  at
 
 
 org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
  at
 
 
 org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
  at java.lang.Thread.run(Thread.java:662)
  Dec 9, 2013 1:17:05 PM
  org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor
 getLoad
  SEVERE: The required nodeId is not available: kush-11 :null
  java.lang.NullPointerException
  at
 
 
 org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
  at
 
 
 org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
  at
 
 
 org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
  at java.lang.Thread.run(Thread.java:662)
 
 
  I get this error message for all 44 of the nodes I have defined in my
  nodes.xml config.
 
 
  A couple questions:
 
  1.  Can I simply run resource manager without the Plugin via a change in
  config?  I tried to comment out the config lines, but it didn't work.
 
  2.  If I cannot simply disconnect the Ganglia Dependency, then do I
 have to
  setup and run a Ganglia metadata daemon?
 
 
  I look forward to your reply,
 
 
  Cameron
 
  --
 
  Sent from a Tin Can attached to a String
 

Re: [Ganglia plugin] Next steps

[Rajith Siriwardana]

Hi all,

Attached/linked diagram [1] shows how the GangliaResourceMonitorFactory
will be integrated to AssignmentMonitor to calculate load.
In here in AssignmentMonitor it keeps the node's load in a static hashmap
(nodeId, load) so I guess the *loadMap should be updated in a timely
manner* (ex: 1 min interval) by parsing the ganglia XML right?

Since the load we need is not a traditional value and it's a value which
says how many of these jobs can fit on a machine. So as I understood, the
load calculation should happen a way that, which the most relevant metrics
are taken into calculation and weights should be added to the values. then
the load value should normalize within the range of 0 and 1.
I guess following metrics are the most relevant ones with the default
Ganglia metrics for the calculation.

load_one = one minute load average
load_five = five minutes load average
load_fifteen = fifteen minutes load average

mem_free = amount of available memory
swap_free = amount of available swap memory

Followings are the models currently have in mind.
(I). weight the 1 min, 5 min and 15 min load numbers and normalize the
value.
(II). adding the mem_free and swap_free metrics to the calculation with
model I.

more weight should goes to either 5 or 15. according to [3].
#1. *but how can I rationalize the weights i give?*
#2. furthermore what is the capacity of a Node? since we are talking
about *normalization
what is the role of this capacity?* how it affects this calculation. (when
assigning load to a particular node it calculate something like if
(loadValue = (loadCap - curLoad)) inhere loadCap = node.getCapacity() and
curLoad=loadMap.get(node.getNodeId())).intValue() )

Other considerations
#3. what should be the value if the node is offline?

We can say a particular Node is offline by TN and TMAX value. gmetad, a
host is considered offline and is ignored if TN  4 * TMAX.[2]

(TN :  TN value is the number of seconds since the metric was last
updated TMAX:
The maximum time in seconds between gmetric calls)

*default  ganglia metrics is listed here and your thoughts are welcome.*
disk_free = Disk Space Available
machine_type = System architecture
bytes_out = Number of bytes out per second
gexec = DESC VAL = gexec available
proc_total = Total number of processes
cpu_nice = Percentage of CPU utilization that occurred while executing at
the user level with nice priority
pkts_in = Packets in per second
cpu_speed = CPU Speed in terms of MHz
boottime = The last time that the system was started
cpu_wio = Percentage of time that the CPU or CPUs were idle during which
the system had an outstanding disk I/O request
os_name = Operating system name
load_one = One minute load average
os_release = Operating system release date
disk_total = Total available disk space
cpu_user = Percentage of CPU utilization that occurred while executing at
the user level
cpu_idle = Percentage of time that the CPU or CPUs were idle and the system
did not have an outstanding disk I/O request
swap_free = Amount of available swap memory
mem_cached = Amount of cached memory
pkts_out = Packets out per second
load_five = Five minute load average
cpu_num = Total number of CPUs
load_fifteen  = Fifteen minute load average
mem_free = Amount of available memory
cpu_system = Percentage of CPU utilization that occurred while executing at
the system level
proc_run = Total number of running processes
mem_total = Total amount of memory displayed in KBs
cpu_aidle = Percent of time since boot idle CPU
bytes_in  = Number of bytes in per second
mem_buffers  = Amount of buffered memory
mem_shared = Amount of shared memory
swap_total = Total amount of swap space displayed in KBs
part_max_used = Maximum percent used for all partitions

[1] https://issues.apache.org/jira/secure/attachment/12589911/diagram1.png
[2] http://entropy.gforge.inria.fr/ganglia.html
[3] http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages


Cheers,
Rajith



On Fri, Jun 21, 2013 at 7:22 PM, Rajith Siriwardana 
rajithsiriward...@gmail.com wrote:


 moving the conversation to dev.

 Cheers,
 Rajith

 On Thu, Jun 20, 2013 at 11:10 AM, Chris Mattmann chris.mattm...@gmail.com
  wrote:

 Hi Rajith,

 RE: #1 yep that's the next step.

 RE: #2, I would create a pluggable function/class that allows
 different Besting algorithms to be plugged in. One simple one
 would be AverageLoad (avg between the 3 load values). Another
 simple would be FiveMinuteLoad; another OneMinLoad; etc. I would
 also imagine allowing ArbitraryMetricWeightedAvgLoad where it takes
 in maybe a ListString specifying the metric names, and then also
 maybe a HashMapString, Double that identifies the metric name,
 and then the weight to apply in the weighted average, e.g., maybe
 {{1minload, 3.0}, {5minload, 10.0}, {15minload, 1.0}}

 indicating that the final load should be calculated as:

 3*[val of 1minLoad] + 10*[val of 5minLoad] + 1*[val of 15minLoad

[Ganglia plugin OODT-219] Command line presentation (draft)

[Rajith Siriwardana]

Following draft categorization, shows how the CLI commands and responses
should be implemented. Grid info:

   -

   args: gmetad host, port, “--gridinfo” /“--summary”



   -

   response :

summarized clusters: per cluster (name, owner, latitude  longitude, online
Nodes, offline Nodes)


Cluster info:

   -

   args : gmetad host, port, “--clusterinfo -name cluster name”

(multiple clusters may be there by the same name, in that case all the same
name clusters)


   -

   response:

   Cluster name, owner, latitude  longitude, url, local time

   Total CPUs, Total Disk, Total Memory

   summarized Nodes : per node (name, IP,location, offline/online, load,
   memory_free, disk_free)


   (We can say a particular Node is offline by TN and TMAX value. gmetad, a
   host is considered offline and is ignored if TN  4 * TMAX.

   source: http://entropy.gforge.inria.fr/ganglia.html)

   TN :  TN value is the number of seconds since the metric was last updated

   TMAX: The maximum time in seconds between gmetric calls


Node info:

From Node ID and ip we should be able to get status of a node.  I guess
OODT “Node Id”  and the ganglia host name are the same.


   - args: gmetad host, port, “-nodeId node id”
   - args: gmetad host, port, “-nodeurl node url”
   - response: Node info: Name, IP,(online/offline), reported, location,
   gmond started, all 32 default metrics + added metrics


Suggestions and your advices are warmly welcome.

Thank you,
Rajith

Re: [Ganglia plugin OODT-219] Command line presentation (draft)

[Rajith Siriwardana]

Sorry for the large text, it was a mistake.


On Sun, Jun 16, 2013 at 11:47 PM, Rajith Siriwardana 
rajithsiriward...@gmail.com wrote:

 Following draft categorization, shows how the CLI commands and responses
 should be implemented. Grid info:

-

args: gmetad host, port, “--gridinfo” /“--summary”



-

response :

 summarized clusters: per cluster (name, owner, latitude  longitude,
 online Nodes, offline Nodes)


 Cluster info:

-

args : gmetad host, port, “--clusterinfo -name cluster name”

 (multiple clusters may be there by the same name, in that case all the
 same name clusters)


-

response:

Cluster name, owner, latitude  longitude, url, local time

Total CPUs, Total Disk, Total Memory

summarized Nodes : per node (name, IP,location, offline/online, load,
memory_free, disk_free)


(We can say a particular Node is offline by TN and TMAX value. gmetad,
a host is considered offline and is ignored if TN  4 * TMAX.

source: http://entropy.gforge.inria.fr/ganglia.html)

TN :  TN value is the number of seconds since the metric was last
updated

TMAX: The maximum time in seconds between gmetric calls


 Node info:

 From Node ID and ip we should be able to get status of a node.  I guess
 OODT “Node Id”  and the ganglia host name are the same.


- args: gmetad host, port, “-nodeId node id”
- args: gmetad host, port, “-nodeurl node url”
- response: Node info: Name, IP,(online/offline), reported, location,
gmond started, all 32 default metrics + added metrics


 Suggestions and your advices are warmly welcome.

 Thank you,
 Rajith

Review Request: OODT-618 [sub task of OODT-219] : Ganglia XML parser for Ganglia plugin

[Rajith Siriwardana]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11819/
---

Review request for oodt and ChrisRK ChrisRK.


Description
---

This patch is a subtask of Ganglia plugin [OODT-219] and I have added a SAX 
parser to parse the XML dump which will be downloaded to Ganglia plugin and 
ganglia metric builder. other than that this patch contains test scenarios to 
test the functionality and to assure the XML tag parsing, sample ganglia xml 
dump also with this patch.


This addresses bug OODT-618.
https://issues.apache.org/jira/browse/OODT-618


Diffs
-

  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/exceptions/MonitoringException.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaMetKeys.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaXMLParser.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Cluster.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Host.java
 PRE-CREATION 
  
trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Metric.java
 PRE-CREATION 
  
trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestGangliaXMLParser.java
 PRE-CREATION 
  trunk/resource/src/testdata/gangliaXMLdump.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/11819/diff/


Testing
---

Test scenarios added to test the parsing and to assure XML tags.


Thanks,

Rajith Siriwardana

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Thanks and congratulations for you too Ross !!

Cheers,
Rajith


On Tue, May 28, 2013 at 5:27 PM, Ross Laidlaw rlaidlaw.o...@gmail.comwrote:

 Congratulations Rajith!

 Nice work on your project so far and well done for gaining a place on GSoC.

 I've also been accepted this year and I'd like to echo your comments to say
 a massive thank you to Chris and Rishi for giving me this opportunity.

 Looking forward to a great summer!

 Ross



 On 28 May 2013 01:55, Rajith Siriwardana rajithsiriward...@ieee.org
 wrote:

  Hi all,
 
  I have been notified that my proposal was accepted for GSOC 2013. Thank
 you
  so much to everybody in the OODT community for your encouragements, kind
  words and welcoming me to your community. Most importantly thank you very
  much Chris for deciding my proposal is worth sharing your valuable time
  during the summer. I'm very happy to have this opportunity to work with
  you.
 
  I'm hoping to have a very productive summer and make this project a
  success.
 
  Thanks
  Rajith
 
 
  T.W.R.O. Siriwardana
  StMIEEE
  Undergraduate
  Department of Computer Science  Engineering
  Faculty of Engineering
  University of Moratuwa
  Sri Lanka.
 
 
  On Tue, Apr 23, 2013 at 9:23 PM, Mattmann, Chris A (398J) 
  chris.a.mattm...@jpl.nasa.gov wrote:
 
   Awesome work!
  
   ++
   Chris Mattmann, Ph.D.
   Senior Computer Scientist
   NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
   Office: 171-266B, Mailstop: 171-246
   Email: chris.a.mattm...@nasa.gov
   WWW:  http://sunset.usc.edu/~mattmann/
   ++
   Adjunct Assistant Professor, Computer Science Department
   University of Southern California, Los Angeles, CA 90089 USA
   ++
  
  
  
  
  
  
   -Original Message-
   From: Rajith Siriwardana rajithsiriward...@ieee.org
   Date: Tuesday, April 23, 2013 8:38 AM
   To: dev dev@oodt.apache.org, jpluser chris.a.mattm...@jpl.nasa.gov
   Subject: Re: Interested in participating GSoC 2013 with OODT
  
   Hi Chris,
   
   I've submitted a first draft of my GSoC 2013 proposal. Here's a link
 to
   the proposal:[1]
   If you'd like me to make any amendments or additions,
   please let me know and I'll update it.
   Many thanks for your valuable time spent on reviewing my proposal in
 the
   first place.[1]
   
  
 
 https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/ra
   jithsiriwardana/1Cheers,
   Rajith
   ---
   T.W.R.O. Siriwardana
   StMIEEE
   Undergraduate
   Department of Computer Science  Engineering
   Faculty of Engineering
   University of Moratuwa
   Sri Lanka.
   
   
   
   
   
   
   On Sat, Apr 20, 2013 at 11:59 AM, Mattmann, Chris A (398J)
   chris.a.mattm...@jpl.nasa.gov wrote:
   
   Thanks Rajith!
   
   ++
   Chris Mattmann, Ph.D.
   Senior Computer Scientist
   NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
   Office: 171-266B, Mailstop: 171-246
   Email: chris.a.mattm...@nasa.gov
   WWW:  http://sunset.usc.edu/~mattmann/
   ++
   Adjunct Assistant Professor, Computer Science Department
   University of Southern California, Los Angeles, CA 90089 USA
   ++
   
   
   
   
   
   
   
   -Original Message-
   From: Rajith Siriwardana rajithsiriward...@gmail.com
   Reply-To: dev@oodt.apache.org dev@oodt.apache.org
   
   Date: Friday, April 12, 2013 10:38 AM
   To: dev dev@oodt.apache.org
   Subject: Re: Interested in participating GSoC 2013 with OODT
   
   On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) 
   chris.a.mattm...@jpl.nasa.gov wrote:
   
Hi Rajith,
   
Thanks, replies below!
   
   
   
-Original Message-
From: Rajith Siriwardana rajithsiriward...@ieee.org
Reply-To: dev@oodt.apache.org dev@oodt.apache.org
Date: Friday, April 12, 2013 10:17 AM
To: dev dev@oodt.apache.org
Subject: Re: Interested in participating GSoC 2013 with OODT
   
Hi Chris,
please find the comments inline.
[..snip..]


Do you know of one? I looked way back in 2005 but there wasn't one
   back
 then.


Still I haven't found a parser compatible with the Apache license
   v2.0,
but
I found some parsers to get the idea of how it should be. I guess
 it
   won't
be hard.
   
+1, great.
   




- In job status monitoring it will involve jvm monitoring
  (that
   is
not
related to the plugin as it just polling gmetad) correct me
 if
   i¹m
misunderstood the job concept.

 Job status is resource manager job status polling. Have you seen
  the
 Job class in resource manager?


yep I did that, gmetad has

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Hi all,

I have been notified that my proposal was accepted for GSOC 2013. Thank you
so much to everybody in the OODT community for your encouragements, kind
words and welcoming me to your community. Most importantly thank you very
much Chris for deciding my proposal is worth sharing your valuable time
during the summer. I'm very happy to have this opportunity to work with you.

I'm hoping to have a very productive summer and make this project a
success.

Thanks
Rajith


T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.


On Tue, Apr 23, 2013 at 9:23 PM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Awesome work!

 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++






 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Date: Tuesday, April 23, 2013 8:38 AM
 To: dev dev@oodt.apache.org, jpluser chris.a.mattm...@jpl.nasa.gov
 Subject: Re: Interested in participating GSoC 2013 with OODT

 Hi Chris,
 
 I've submitted a first draft of my GSoC 2013 proposal. Here's a link to
 the proposal:[1]
 If you'd like me to make any amendments or additions,
 please let me know and I'll update it.
 Many thanks for your valuable time spent on reviewing my proposal in the
 first place.[1]
 
 https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/ra
 jithsiriwardana/1Cheers,
 Rajith
 ---
 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.
 
 
 
 
 
 
 On Sat, Apr 20, 2013 at 11:59 AM, Mattmann, Chris A (398J)
 chris.a.mattm...@jpl.nasa.gov wrote:
 
 Thanks Rajith!
 
 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++
 
 
 
 
 
 
 
 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@gmail.com
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 
 Date: Friday, April 12, 2013 10:38 AM
 To: dev dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT
 
 On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) 
 chris.a.mattm...@jpl.nasa.gov wrote:
 
  Hi Rajith,
 
  Thanks, replies below!
 
 
 
  -Original Message-
  From: Rajith Siriwardana rajithsiriward...@ieee.org
  Reply-To: dev@oodt.apache.org dev@oodt.apache.org
  Date: Friday, April 12, 2013 10:17 AM
  To: dev dev@oodt.apache.org
  Subject: Re: Interested in participating GSoC 2013 with OODT
 
  Hi Chris,
  please find the comments inline.
  [..snip..]
  
  
  Do you know of one? I looked way back in 2005 but there wasn't one
 back
   then.
  
  
  Still I haven't found a parser compatible with the Apache license
 v2.0,
  but
  I found some parsers to get the idea of how it should be. I guess it
 won't
  be hard.
 
  +1, great.
 
  
  
  
  
  - In job status monitoring it will involve jvm monitoring (that
 is
  not
  related to the plugin as it just polling gmetad) correct me if
 i¹m
  misunderstood the job concept.
  
   Job status is resource manager job status polling. Have you seen the
   Job class in resource manager?
  
  
  yep I did that, gmetad has nothing to do with polling job status
  information. Should GangliaAssignmentMonitor worry about obtaining job
  status information then? am I missing something here?
 
  I'm not sure what you mean about it not having anything to do with job
  polling?
  gmetad collects metrics published by gmond. Some of those metrics are
  related to
  CPU load, disk space, etc. I guess that's what I meant, more than job
  status.
 
 
 No Chris I guess the confusion caused by this
 
   In the issue OODT-219 it says Write a GangliaAssignmentMonitor and
 
 
 GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
 status information*. hence I thought it should also poll Job status
 information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
 resource node monitoring.
 
 Thanks for the clarification Chris.
 
 Thanks

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

hey Chris,

That's great, I'll create a draft proposal and share it with you.

many thanks for your time.

Cheers,
Rajith


T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.


On Wed, Apr 17, 2013 at 8:00 PM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hey Rajith,

 Thanks. Can you start working on a proposal maybe a Google doc,
 and then share with me?

 I'll look at it and try to help out and we can go from there.

 Cheers,
 Chris

 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++






 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 Date: Tuesday, April 16, 2013 11:16 AM
 To: dev dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT

 Hi guys,
 
 Ganglia plugin initial architecture diagram is attached with OODT-219. You
 can find it in [1]. Your advises are warmly welcome to modify it. I guess
 Chris [2] will be mentoring this project. And I wanna know what are
 specifications you guys are expecting in the project proposal except the
 things specified in [3].
 
 [1]
 
 https://issues.apache.org/jira/secure/attachment/12578972/architecture_dia
 gram.jpg
 [2]
 https://issues.apache.org/jira/secure/ViewProfile.jspa?name=chrismattmann
 [3] http://community.apache.org/gsoc.html
 
 
 
 Thank you,
 
 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.
 
 
 On Fri, Apr 12, 2013 at 11:08 PM, Rajith Siriwardana 
 rajithsiriward...@gmail.com wrote:
 
 
 
 
  On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) 
  chris.a.mattm...@jpl.nasa.gov wrote:
 
  Hi Rajith,
 
  Thanks, replies below!
 
 
 
  -Original Message-
  From: Rajith Siriwardana rajithsiriward...@ieee.org
  Reply-To: dev@oodt.apache.org dev@oodt.apache.org
  Date: Friday, April 12, 2013 10:17 AM
  To: dev dev@oodt.apache.org
  Subject: Re: Interested in participating GSoC 2013 with OODT
 
  Hi Chris,
  please find the comments inline.
  [..snip..]
  
  
  Do you know of one? I looked way back in 2005 but there wasn't one
 back
   then.
  
  
  Still I haven't found a parser compatible with the Apache license
 v2.0,
  but
  I found some parsers to get the idea of how it should be. I guess it
  won't
  be hard.
 
  +1, great.
 
  
  
  
  
  - In job status monitoring it will involve jvm monitoring (that
 is
  not
  related to the plugin as it just polling gmetad) correct me if
 i¹m
  misunderstood the job concept.
  
   Job status is resource manager job status polling. Have you seen the
   Job class in resource manager?
  
  
  yep I did that, gmetad has nothing to do with polling job status
  information. Should GangliaAssignmentMonitor worry about obtaining job
  status information then? am I missing something here?
 
  I'm not sure what you mean about it not having anything to do with job
  polling?
  gmetad collects metrics published by gmond. Some of those metrics are
  related to
  CPU load, disk space, etc. I guess that's what I meant, more than job
  status.
 
 
  No Chris I guess the confusion caused by this
 
In the issue OODT-219 it says Write a GangliaAssignmentMonitor
 and
  GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
  status information*. hence I thought it should also poll Job status
  information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
  resource node monitoring.
 
  Thanks for the clarification Chris.
 
  Thanks for clarifying.
 
  Cheers,
  Chris
 
  ++
  Chris Mattmann, Ph.D.
  Senior Computer Scientist
  NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
  Office: 171-266B, Mailstop: 171-246
  Email: chris.a.mattm...@nasa.gov
  WWW:  http://sunset.usc.edu/~mattmann/
  ++
  Adjunct Assistant Professor, Computer Science Department
  University of Southern California, Los Angeles, CA 90089 USA
  ++
 
 
 
 
 

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Hi guys,

Ganglia plugin initial architecture diagram is attached with OODT-219. You
can find it in [1]. Your advises are warmly welcome to modify it. I guess
Chris [2] will be mentoring this project. And I wanna know what are
specifications you guys are expecting in the project proposal except the
things specified in [3].

[1]
https://issues.apache.org/jira/secure/attachment/12578972/architecture_diagram.jpg
[2]
https://issues.apache.org/jira/secure/ViewProfile.jspa?name=chrismattmann
[3] http://community.apache.org/gsoc.html



Thank you,

T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.


On Fri, Apr 12, 2013 at 11:08 PM, Rajith Siriwardana 
rajithsiriward...@gmail.com wrote:




 On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) 
 chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Rajith,

 Thanks, replies below!



 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 Date: Friday, April 12, 2013 10:17 AM
 To: dev dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT

 Hi Chris,
 please find the comments inline.
 [..snip..]
 
 
 Do you know of one? I looked way back in 2005 but there wasn't one back
  then.
 
 
 Still I haven't found a parser compatible with the Apache license v2.0,
 but
 I found some parsers to get the idea of how it should be. I guess it
 won't
 be hard.

 +1, great.

 
 
 
 
 - In job status monitoring it will involve jvm monitoring (that is
 not
 related to the plugin as it just polling gmetad) correct me if i¹m
 misunderstood the job concept.
 
  Job status is resource manager job status polling. Have you seen the
  Job class in resource manager?
 
 
 yep I did that, gmetad has nothing to do with polling job status
 information. Should GangliaAssignmentMonitor worry about obtaining job
 status information then? am I missing something here?

 I'm not sure what you mean about it not having anything to do with job
 polling?
 gmetad collects metrics published by gmond. Some of those metrics are
 related to
 CPU load, disk space, etc. I guess that's what I meant, more than job
 status.


 No Chris I guess the confusion caused by this

   In the issue OODT-219 it says Write a GangliaAssignmentMonitor and
 GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
 status information*. hence I thought it should also poll Job status
 information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
 resource node monitoring.

 Thanks for the clarification Chris.

 Thanks for clarifying.

 Cheers,
 Chris

 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Hi Chris,
please find the comments inline.

Thanks
On Fri, Apr 12, 2013 at 10:56 AM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Rajith,


 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 Date: Sunday, April 7, 2013 9:24 AM
 To: dev dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT

 hi all,
 
 *
 
 As I understood there will be resource node monitoring and Job status
 monitoring happen using ganglia. Resource management components plugin
 will
 poll gmetad periodically by xml over tcp and receive an xml file of the
 last status monitored.

 +1.

 
 I need some clarification on following matters
 
 
-
 
By polling gmetad you will get aggregated monitoring data of a cluster.
by listening gmond multicast/unicast channel you can get xml from a
 gmond
running in a node. I guess it should be polling gmetad.

 Yep -- we can create a Java socket connection to port 8649 I believe to
 get the XML document. So we'll need to write a Java parser for its schema
 if there isn't one already that we can use.


Yes, that is what I have in mind.

Do you know of one? I looked way back in 2005 but there wasn't one back
 then.


Still I haven't found a parser compatible with the Apache license v2.0, but
I found some parsers to get the idea of how it should be. I guess it won't
be hard.




- In job status monitoring it will involve jvm monitoring (that is not
related to the plugin as it just polling gmetad) correct me if i¹m
misunderstood the job concept.

 Job status is resource manager job status polling. Have you seen the
 Job class in resource manager?


yep I did that, gmetad has nothing to do with polling job status
information. Should GangliaAssignmentMonitor worry about obtaining job
status information then? am I missing something here?



-
 
Is it only be able to poll default gmond metrics. or should it be
pluggable with gmetric functionality which is able to introduce
 additional
metrics.

 Yep it should plug into the Resource Manager AssignmentMonitor interface.

 Does that make sense?

 Cheers,
 Chris

 
 Anything you need to clarify or I have misunderstood?*
 
 Thank you,
 
 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.
 
 
 On Sun, Mar 31, 2013 at 7:05 AM, Rajith Siriwardana 
 rajithsiriward...@ieee.org wrote:
 
  Awesome! thanks Chris.
 
  Thank you,
 
  T.W.R.O. Siriwardana
  StMIEEE
  Undergraduate
  Department of Computer Science  Engineering
  Faculty of Engineering
   University of Moratuwa
  Sri Lanka.
 
  On Sun, Mar 31, 2013 at 2:27 AM, Mattmann, Chris A (388J) 
  chris.a.mattm...@jpl.nasa.gov wrote:
 
  Hi Rajith,
 
  We're looking forward to you participating.
 
  I just outlined the process to someone else who is trying to
 participate
  in Apache Mesos through GSOC 2013, so I'll repaste here:
 
 
1.  ASF is accepted/declined as GSoC 2013 org (let's hope it's
 accepted
  :) )
2.  Rajith talks with OODT mentor and OODT mentor agrees to mentor
3. Rajith comes up with proposal, referencing the GSoC project and
 JIRA
  issue, inputs into
  Google Melange and inputs into Google Melange for GSOC 2013
  http://www.google-melange.com/gsoc/homepage/google/gsoc2013
4.  OODT mentor signs up for Melange
  http://www.google-melange.com/gsoc/homepage/google/gsoc2013
  and lists ASF as organization
5. Rajith requests OODT mentor as mentor (ASF GSoC 2013 admins
  approve/decline proposed project)
6.  Rajith and OODT mentor find out if the project was approved
7.  Project happens
8. OODT mentor and Rajith must coordinate, and then work together
  through the process.
 
 
  HTH!
 
  Cheers,
  Chris
 
  ++
  Chris Mattmann, Ph.D.
  Senior Computer Scientist
  NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
  Office: 171-266B, Mailstop: 171-246
  Email: chris.a.mattm...@nasa.gov
  WWW:  http://sunset.usc.edu/~mattmann/
  ++
  Adjunct Assistant Professor, Computer Science Department
  University of Southern California, Los Angeles, CA 90089 USA
  ++
 
 
 
 
 
 
  -Original Message-
  From: Rajith Siriwardana rajithsiriward...@ieee.org
  Reply-To: dev@oodt.apache.org dev@oodt.apache.org
  Date: Wednesday, March 27, 2013 9:08 AM
  To: dev@oodt.apache.org dev@oodt.apache.org
  Subject: Interested in participating GSoC 2013 with OODT
 
  hi,
  I'm a 4th year (final year) undergraduate of Faculty of Engineering
  University of Moratuwa majoring in Computer Science and Engineering
 and I
  would like contribute to OODT.
  I did my internship last year which spanned for 9 months in WSO2[1].
  which

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Rajith,

 Thanks, replies below!



 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 Date: Friday, April 12, 2013 10:17 AM
 To: dev dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT

 Hi Chris,
 please find the comments inline.
 [..snip..]
 
 
 Do you know of one? I looked way back in 2005 but there wasn't one back
  then.
 
 
 Still I haven't found a parser compatible with the Apache license v2.0,
 but
 I found some parsers to get the idea of how it should be. I guess it won't
 be hard.

 +1, great.

 
 
 
 
 - In job status monitoring it will involve jvm monitoring (that is
 not
 related to the plugin as it just polling gmetad) correct me if i¹m
 misunderstood the job concept.
 
  Job status is resource manager job status polling. Have you seen the
  Job class in resource manager?
 
 
 yep I did that, gmetad has nothing to do with polling job status
 information. Should GangliaAssignmentMonitor worry about obtaining job
 status information then? am I missing something here?

 I'm not sure what you mean about it not having anything to do with job
 polling?
 gmetad collects metrics published by gmond. Some of those metrics are
 related to
 CPU load, disk space, etc. I guess that's what I meant, more than job
 status.


No Chris I guess the confusion caused by this

  In the issue OODT-219 it says Write a GangliaAssignmentMonitor and
GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
status information*. hence I thought it should also poll Job status
information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
resource node monitoring.

Thanks for the clarification Chris.

Thanks for clarifying.

 Cheers,
 Chris

 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Awesome! thanks Chris.

Thank you,

T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.

On Sun, Mar 31, 2013 at 2:27 AM, Mattmann, Chris A (388J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Rajith,

 We're looking forward to you participating.

 I just outlined the process to someone else who is trying to participate
 in Apache Mesos through GSOC 2013, so I'll repaste here:


   1.  ASF is accepted/declined as GSoC 2013 org (let's hope it's accepted
 :) )
   2.  Rajith talks with OODT mentor and OODT mentor agrees to mentor
   3. Rajith comes up with proposal, referencing the GSoC project and JIRA
 issue, inputs into
 Google Melange and inputs into Google Melange for GSOC 2013
 http://www.google-melange.com/gsoc/homepage/google/gsoc2013
   4.  OODT mentor signs up for Melange
 http://www.google-melange.com/gsoc/homepage/google/gsoc2013
 and lists ASF as organization
   5. Rajith requests OODT mentor as mentor (ASF GSoC 2013 admins
 approve/decline proposed project)
   6.  Rajith and OODT mentor find out if the project was approved
   7.  Project happens
   8. OODT mentor and Rajith must coordinate, and then work together
 through the process.


 HTH!

 Cheers,
 Chris

 ++
 Chris Mattmann, Ph.D.
 Senior Computer Scientist
 NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
 Office: 171-266B, Mailstop: 171-246
 Email: chris.a.mattm...@nasa.gov
 WWW:  http://sunset.usc.edu/~mattmann/
 ++
 Adjunct Assistant Professor, Computer Science Department
 University of Southern California, Los Angeles, CA 90089 USA
 ++






 -Original Message-
 From: Rajith Siriwardana rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.org dev@oodt.apache.org
 Date: Wednesday, March 27, 2013 9:08 AM
 To: dev@oodt.apache.org dev@oodt.apache.org
 Subject: Interested in participating GSoC 2013 with OODT

 hi,
 I'm a 4th year (final year) undergraduate of Faculty of Engineering
 University of Moratuwa majoring in Computer Science and Engineering and I
 would like contribute to OODT.
 I did my internship last year which spanned for 9 months in WSO2[1]. which
 is a middle-ware company and I worked with their WSO2 Stratos product[2].
 Basically I was assigned for developing WSO2 Stratos system level health
 monitoring component and WSO2 Stratos heartbeat monitoring package.  For
 WSO2 Stratos health monitoring component[3] before finalizing Nagios[4]
 for
 monitoring with mk_livestatus[5] replacing NagiosXML engine for query
 stats, which is shown in the architecture image, I did some work with
 Ganglia (how to use it for WSO2 Stratos system level health monitoring).
 Furthermore I took part in test automation hackathon carried out for WSO2
 ESB[6] around 2 months.
 So I believe I have enough knowledge to complete the *GSoC 2013 project
  Monitor that plugs into ganglia
 OODT-219https://issues.apache.org/jira/browse/OODT-219
  [7]* with some guidance.
 Please give me any advice, guidance you'd like to extend.
 You can access my Linkedin profile via [8]
 
 [1]http://wso2.com/
 [2]http://wso2.com/cloud/stratos/
 [3]http://mail.wso2.org/mailarchive/architecture/2012-June/008827.html
 [4]http://www.nagios.org/
 [5]http://mathias-kettner.de/checkmk_livestatus.html
 [6]http://wso2.com/products/enterprise-service-bus/
 [7]https://issues.apache.org/jira/browse/OODT-219
 [8]http://lk.linkedin.com/in/rajithsiriwardana/
 
 Thank you,
 
 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

Thanks guys, me too looking forward to work with you guys.


Thank you,

T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.



On Thu, Mar 28, 2013 at 9:14 AM, Mattmann, Chris A (388J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Rajith!

 Thank you for your interest in OODT-219! Would love to have you help out
 on the project.

 The mailing list doesn't accept attachments, so can you head over the
 OODT-219 and attach
 the diagram there?


I have attached the diagram to OODT-219.



 Thank you and looking forward to working with you on this!

 Cheers,
 Chris

 From: Rajith Siriwardana rajithsiriward...@ieee.orgmailto:
 rajithsiriward...@ieee.org
 Reply-To: dev@oodt.apache.orgmailto:dev@oodt.apache.org 
 dev@oodt.apache.orgmailto:dev@oodt.apache.org
 Date: Wednesday, March 27, 2013 9:20 AM
 To: dev dev@oodt.apache.orgmailto:dev@oodt.apache.org
 Subject: Re: Interested in participating GSoC 2013 with OODT

 hi
 small mistake, forgot to attach the high level deployment architecture
 diagram which i was talking about. note that this architecture diagram
 changed to use mk_livestatus replacing Nagios XML engine to query stats. I
 can provide more details upon request  about the the project I carried out.

 Thank you,

 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.


 On Wed, Mar 27, 2013 at 9:38 PM, Rajith Siriwardana 
 rajithsiriward...@ieee.orgmailto:rajithsiriward...@ieee.org wrote:
 hi,
 I'm a 4th year (final year) undergraduate of Faculty of Engineering
 University of Moratuwa majoring in Computer Science and Engineering and I
 would like contribute to OODT.
 I did my internship last year which spanned for 9 months in WSO2[1]. which
 is a middle-ware company and I worked with their WSO2 Stratos product[2].
 Basically I was assigned for developing WSO2 Stratos system level health
 monitoring component and WSO2 Stratos heartbeat monitoring package.  For
 WSO2 Stratos health monitoring component[3] before finalizing Nagios[4] for
 monitoring with mk_livestatus[5] replacing NagiosXML engine for query
 stats, which is shown in the architecture image, I did some work with
 Ganglia (how to use it for WSO2 Stratos system level health monitoring).
 Furthermore I took part in test automation hackathon carried out for WSO2
 ESB[6] around 2 months.
 So I believe I have enough knowledge to complete the GSoC 2013 project
  Monitor that plugs into ganglia OODT-219
 https://issues.apache.org/jira/browse/OODT-219 [7] with some guidance.
 Please give me any advice, guidance you'd like to extend.
 You can access my Linkedin profile via [8]

 [1]http://wso2.com/
 [2]http://wso2.com/cloud/stratos/
 [3]http://mail.wso2.org/mailarchive/architecture/2012-June/008827.html
 [4]http://www.nagios.org/
 [5]http://mathias-kettner.de/checkmk_livestatus.html
 [6]http://wso2.com/products/enterprise-service-bus/
 [7]https://issues.apache.org/jira/browse/OODT-219
 [8]http://lk.linkedin.com/in/rajithsiriwardana/

 Thank you,

 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
 University of Moratuwa
 Sri Lanka.

Re: Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

hi
small mistake, forgot to attach the high level deployment architecture
diagram which i was talking about. note that this architecture diagram
changed to use mk_livestatus replacing Nagios XML engine to query stats. I
can provide more details upon request  about the the project I carried out.

Thank you,

T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
 University of Moratuwa
Sri Lanka.


On Wed, Mar 27, 2013 at 9:38 PM, Rajith Siriwardana 
rajithsiriward...@ieee.org wrote:

 hi,
 I'm a 4th year (final year) undergraduate of Faculty of Engineering
 University of Moratuwa majoring in Computer Science and Engineering and I
 would like contribute to OODT.
 I did my internship last year which spanned for 9 months in WSO2[1]. which
 is a middle-ware company and I worked with their WSO2 Stratos product[2].
 Basically I was assigned for developing WSO2 Stratos system level health
 monitoring component and WSO2 Stratos heartbeat monitoring package.  For
 WSO2 Stratos health monitoring component[3] before finalizing Nagios[4] for
 monitoring with mk_livestatus[5] replacing NagiosXML engine for query
 stats, which is shown in the architecture image, I did some work with
 Ganglia (how to use it for WSO2 Stratos system level health monitoring).
 Furthermore I took part in test automation hackathon carried out for WSO2
 ESB[6] around 2 months.
 So I believe I have enough knowledge to complete the *GSoC 2013 project
  Monitor that plugs into ganglia 
 OODT-219https://issues.apache.org/jira/browse/OODT-219
  [7]* with some guidance.
 Please give me any advice, guidance you'd like to extend.
 You can access my Linkedin profile via [8]

 [1]http://wso2.com/
 [2]http://wso2.com/cloud/stratos/
 [3]http://mail.wso2.org/mailarchive/architecture/2012-June/008827.html
 [4]http://www.nagios.org/
 [5]http://mathias-kettner.de/checkmk_livestatus.html
 [6]http://wso2.com/products/enterprise-service-bus/
 [7]https://issues.apache.org/jira/browse/OODT-219
 [8]http://lk.linkedin.com/in/rajithsiriwardana/

 Thank you,

 T.W.R.O. Siriwardana
 StMIEEE
 Undergraduate
 Department of Computer Science  Engineering
 Faculty of Engineering
  University of Moratuwa
 Sri Lanka.

Interested in participating GSoC 2013 with OODT

[Rajith Siriwardana]

hi,
I'm a 4th year (final year) undergraduate of Faculty of Engineering
University of Moratuwa majoring in Computer Science and Engineering and I
would like contribute to OODT.
I did my internship last year which spanned for 9 months in WSO2[1]. which
is a middle-ware company and I worked with their WSO2 Stratos product[2].
Basically I was assigned for developing WSO2 Stratos system level health
monitoring component and WSO2 Stratos heartbeat monitoring package.  For
WSO2 Stratos health monitoring component[3] before finalizing Nagios[4] for
monitoring with mk_livestatus[5] replacing NagiosXML engine for query
stats, which is shown in the architecture image, I did some work with
Ganglia (how to use it for WSO2 Stratos system level health monitoring).
Furthermore I took part in test automation hackathon carried out for WSO2
ESB[6] around 2 months.
So I believe I have enough knowledge to complete the *GSoC 2013 project
 Monitor that plugs into ganglia
OODT-219https://issues.apache.org/jira/browse/OODT-219
 [7]* with some guidance.
Please give me any advice, guidance you'd like to extend.
You can access my Linkedin profile via [8]

[1]http://wso2.com/
[2]http://wso2.com/cloud/stratos/
[3]http://mail.wso2.org/mailarchive/architecture/2012-June/008827.html
[4]http://www.nagios.org/
[5]http://mathias-kettner.de/checkmk_livestatus.html
[6]http://wso2.com/products/enterprise-service-bus/
[7]https://issues.apache.org/jira/browse/OODT-219
[8]http://lk.linkedin.com/in/rajithsiriwardana/

Thank you,

T.W.R.O. Siriwardana
StMIEEE
Undergraduate
Department of Computer Science  Engineering
Faculty of Engineering
University of Moratuwa
Sri Lanka.