Re: Filemgr Tests

[Lewis John Mcgibbney]

I'm going to let this stew over the weekend and come up with some comments
of my own Tom.
There is a LOT for discussion.


On Fri, Jul 18, 2014 at 5:21 PM, Tom Barber  wrote:

> Hello OODT Development Team
>
> I was wondering if
> a) it would make sense for us to either have a call at some point, or just
> get a thread/jira started that allows us to get a rough idea of what
> aspects of OODT people are working on, what they need etc as currently I
> don't think many of us have a real clue
> b) On the new website, would it be worth publishing a "roadmap" cause, I
> have no clue where the project is going
> c) Are there any better ways of sorting out, planning, roadmaps, meetings,
> community stuff, and collaboration on tasks, that we can take from other
> Apache projects, because, whilst I enjoy OODT, and the ideas behind it (and
> I appreciate within the NASA bunch there is probably more cohesion), it
> seems a little rudderless in terms of direction. Feel free to flame me at
> this point lovely community people.
>
> Regards
>
> Tom
>



-- 
*Lewis*

Jenkins build is still unstable: oodt-trunk #1399

[Apache Jenkins Server]

See 

Jenkins build is still unstable: oodt-trunk » Catalog and Archive File Management Component #1399

[Apache Jenkins Server]

See 

Jenkins build is back to stable : oodt-trunk » CAS Protocol SFTP Implementation #1399

[Apache Jenkins Server]

See 

Re: Activity Logger in Commons

[kelly]

My pleasure, doctor.

The use case for the o.a.oodt.commons.activity package was to enable 
distributed logging mostly for ascertaining performance of the OODT grid code.

For example, suppose we had an activity like "make some stew" and we suspected 
a bottleneck in it somewhere, or we wanted to see what steps occurred; moreover 
"making stew" turned out to be a distributed activity, involving CookTop, Oven, 
Fridge, etc., all running on different hosts. The effort to make stew involves 
several incidents.

Make some classes:

import o.a.oodt.commons.activity.Incident;
public class BaconRenderedIncident extends Incident { ... }
public class ChickenBrownedIncident extends Incident { ... }
public class VegetablesChoppedIncident extends Incident { ... }
public class WineBottleOppenedIncident extends Incident { ... }
...

Set some properties:

activity.host=some.host.over.there
activity.port=4556

Create an activity factory:

import o.a.oodt.commons.activity.DatagramLoggingActivityFactory;
import o.a.oodt.commons.activity.ActivityStopped;
...
public class Kitchen {
private factory = new DatagramLoggingActivityFactory():
public void makeStew() {
Activity activity = factory.createActivity();
remoteCookTop.makeStew(activity.getID());
remoteChoppingBlock.makeStew(activity.getID());
activity.log(new ActivityStopped());
}
}
pubic class CookTop {
private factory = new DatagramLoggingActivityFactory():
...
public void makePasta() { ... }
public void makeStew(String id) {
Activity activity = factory.createActivity();
activity.setID(id);
renderBacon();
activity.log(new BaconRenderedIncident());
brownChicken();
activity.log(new ChickenBrownedIncident());
...
}
}
public class ChoppingBlock {
private factory = new DatagramLoggingActivityFactory():
...
public void chopFingers() { ... }
public void makeStew(String id) {
Activity activity = factory.createActivity();
activity.setID(id);
chopCarrots();
activity.log(new CarrotsChoppedIncident());
diceOnions();
activity.log(new OnionsDicedIncident());
...
}
}

Somewhere on the local network, start this:

java -Dactivity.port=4556 
-Dactivity.storage=o.a.oodt.commons.activity.StandardOutputStorage 
o.a.oodt.commons.activity.DatagramLogger

Then when you make stew, you'll see on the standard output of the above process 
all of the incidents that happened when you made stew.

You like XML, use XMLStandardOutputStorage. You want a database? Use 
SQLDatabaseStorage. Hosts aren't all on a local network? Define your own 
ActivityFactory. Go wild.

Enjoy
--k




On 2014-07-18, at 3:06 PM, Lewis John Mcgibbney  
wrote:

> Hi Folks (specifically kelly),
> As part of our involvement with the DARPA XDATA [0] program of work, we are
> working with Draper Labs [1] on their Activity Logger (User-ALE) [2].
> We're in the process of porting the native JavaScript API to Java and
> embedding it within OODT for XDATA specific experiments.
> I wonder if someone (NutJob) could possibly descrbe what the /activity
> package in commons [3] does?
> Thanks in advance and BTW have a great weekend folks.
> Lewis
> 
> [0] http://www.darpa.mil/Our_Work/I2O/Programs/XDATA.aspx
> [1] http://www.draper.com/
> [2] http://draperlaboratory.github.io/user-ale/
> [3]
> https://svn.apache.org/repos/asf/oodt/trunk/commons/src/main/java/org/apache/oodt/commons/activity/
> 
> -- 
> *Lewis*

Re: Filemgr Tests

[Tom Barber]

Hey Lewis,

I know that may have come across like a gripe at you, it certainly 
wasn't, just a general moan about the state of the test suite :)


I certainly like anything that feeds back to users regarding commits 
either blame emails, jira comments, flashing red hazard lights, you know 
the usual :)


Hopefully now I have a few weeks of hacking time until baby number 2 
(then I may have even more if it keeps me awake all night ;) ), I know 
you've put a lot of effort into the f**g tests and I don't know what 
state of repair they are in, but if I can help out with getting them 
running let me know.


Regards

Tom

On 18/07/14 12:46, Lewis John Mcgibbney wrote:

Hi Tom,
To be honest I've no idea when filemgr (or more generally OODT) tests
broke... it's been a good while now and we keep the last 50 builds so the
exact date is most likely deep into the event horizonby now.
Anyway's, I agree with your comments. It is only fair for us to keep the
codebase _stable_. Unstable commits should most likely be reverted ASAP.
What I like about OODT though is that review board is heavily in use.
I should also mention that we *may* wish to set up a pre-commit test
configuration where a bot will take our patch on Jira and run a build on
Jenkins then post a comment to the Jenkins issue telling us whether the
patch meets certain criteria, etc. If you want to do this then I can maybe
look into it.
Apart from that I am still butchering the filemgr tests little by little
and will have a review board up soon. I also may get hit by a bus soon, or
get sucked down the plug hole in my hotel room bath. ;)
Lewis



On Fri, Jul 18, 2014 at 3:53 AM, Tom Barber(Alabs) 
wrote:


Shifted some development wishes from private@ to dev@ for Chris ;)

Na I'm only messing, but I do think more care should be taken with commits
vs the test suite because I would like to do the Tika upgrade, fix the
database dialects etc, but I only feel comfortable hacking around with that
stuff if we know the tests work because I don't know enough about the
feature set of the components that I've not used and as such rely on the
tests to tell me what I've borked or not.

Equally should I break a test I'd expect my commits to be reverted and I'd
be asked to fix them before recommitting instead of just putting up with
broken tests.

Lets make it so! ;)

Tom







--
*Tom Barber* | Technical Director

meteorite bi
*T:* +44 20 8133 3730
*W:* www.meteorite.bi | *Skype:* meteorite.consulting
*A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK

Re: Filemgr Tests

[Tom Barber]

Hello OODT Development Team

I was wondering if
a) it would make sense for us to either have a call at some point, or 
just get a thread/jira started that allows us to get a rough idea of 
what aspects of OODT people are working on, what they need etc as 
currently I don't think many of us have a real clue
b) On the new website, would it be worth publishing a "roadmap" cause, I 
have no clue where the project is going
c) Are there any better ways of sorting out, planning, roadmaps, 
meetings, community stuff, and collaboration on tasks, that we can take 
from other Apache projects, because, whilst I enjoy OODT, and the ideas 
behind it (and I appreciate within the NASA bunch there is probably more 
cohesion), it seems a little rudderless in terms of direction. Feel free 
to flame me at this point lovely community people.


Regards

Tom

Activity Logger in Commons

[Lewis John Mcgibbney]

Hi Folks (specifically kelly),
As part of our involvement with the DARPA XDATA [0] program of work, we are
working with Draper Labs [1] on their Activity Logger (User-ALE) [2].
We're in the process of porting the native JavaScript API to Java and
embedding it within OODT for XDATA specific experiments.
I wonder if someone (NutJob) could possibly descrbe what the /activity
package in commons [3] does?
Thanks in advance and BTW have a great weekend folks.
Lewis

[0] http://www.darpa.mil/Our_Work/I2O/Programs/XDATA.aspx
[1] http://www.draper.com/
[2] http://draperlaboratory.github.io/user-ale/
[3]
https://svn.apache.org/repos/asf/oodt/trunk/commons/src/main/java/org/apache/oodt/commons/activity/

-- 
*Lewis*

Re: Filemgr Tests

[Lewis John Mcgibbney]

Hi Tom,
To be honest I've no idea when filemgr (or more generally OODT) tests
broke... it's been a good while now and we keep the last 50 builds so the
exact date is most likely deep into the event horizonby now.
Anyway's, I agree with your comments. It is only fair for us to keep the
codebase _stable_. Unstable commits should most likely be reverted ASAP.
What I like about OODT though is that review board is heavily in use.
I should also mention that we *may* wish to set up a pre-commit test
configuration where a bot will take our patch on Jira and run a build on
Jenkins then post a comment to the Jenkins issue telling us whether the
patch meets certain criteria, etc. If you want to do this then I can maybe
look into it.
Apart from that I am still butchering the filemgr tests little by little
and will have a review board up soon. I also may get hit by a bus soon, or
get sucked down the plug hole in my hotel room bath. ;)
Lewis



On Fri, Jul 18, 2014 at 3:53 AM, Tom Barber(Alabs) 
wrote:

> Shifted some development wishes from private@ to dev@ for Chris ;)
>
> Na I'm only messing, but I do think more care should be taken with commits
> vs the test suite because I would like to do the Tika upgrade, fix the
> database dialects etc, but I only feel comfortable hacking around with that
> stuff if we know the tests work because I don't know enough about the
> feature set of the components that I've not used and as such rely on the
> tests to tell me what I've borked or not.
>
> Equally should I break a test I'd expect my commits to be reverted and I'd
> be asked to fix them before recommitting instead of just putting up with
> broken tests.
>
> Lets make it so! ;)
>
> Tom
>



-- 
*Lewis*

Filemgr Tests

[Tom Barber(Alabs)]

Shifted some development wishes from private@ to dev@ for Chris ;)

Na I'm only messing, but I do think more care should be taken with 
commits vs the test suite because I would like to do the Tika upgrade, 
fix the database dialects etc, but I only feel comfortable hacking 
around with that stuff if we know the tests work because I don't know 
enough about the feature set of the components that I've not used and as 
such rely on the tests to tell me what I've borked or not.


Equally should I break a test I'd expect my commits to be reverted and 
I'd be asked to fix them before recommitting instead of just putting up 
with broken tests.


Lets make it so! ;)

Tom