Hi Chris,
Great questions! Here's my (longish) thoughts. I'm cc'ing the hackystat-dev-l list so
that if I get anything wrong, someone will correct me. :-) In particular, Julie Sakuda
and Austen Ito know more about hackyInstaller than I do, and Mike Paulding knows more
about the hackyExperiment package, so they can double check my comments.
1) We will deploy HackyStat in a student lab environment, running Windows 2000
and
Eclipse 3.1. We plan to use the Eclipse sensor to monitor groups of students
working on
development projects.
However the installation on the lab machines is locked down, so that the
users/students
cannot modify it. They however do have the use of a roaming profile (although
they are
deleted by the students when their profile space gets filled up).
It would seem that hackyinstaller is not suitable for the students to use as
they do
not have write access to the Eclipse folder.
Q1) Can we use hackinstaller on the image to install the Eclipse plug in/s or
does this
set some user specific settings in the Eclipse folder.
Yes, you can use hackyinstaller on the image, and no, there are no user specific settings
in the Eclipse folder. Basically, installation of the Eclipse sensor involves downloads
into two locations:
(a) <eclipse_home>/plugins. This part of the installation can be done 'once' and saved
with the Eclipse image.
(b) <user.home>/.hackystat/ This part of the installation has to be done for each
student in the "roaming profile".
The tricky part here is that the Eclipse installer package currently assumes access to
the <eclipse_home> directory. Austen/Julie: let's talk about the best way to allow this
kind of 'split' installation of the Eclipse sensor (i.e. the eclipse jar files and the
sensor.properties file are maintained independently).
Q2) How do you recommend that we set HackStat's settings in each of the
student's
roaming profiles?
Before I answer that question, let me note that there are basically three kinds of
"settings" that need to be dealt with:
(a) Registration. Each student needs to be provided with an account on the Hackystat
server. This involves providing the server with their email address, which will generate
a unique 12 character "user key" for each of them. This key is used by their Eclipse
sensor to identify the owner of the data when it is sent to the server, and also by users
when they log in to the server to see their analyses.
(b) Client-side "sensor properties" setup. Each student's "roaming profile" needs to
have a <user.home>/.hackystat directory. This directory is created and most of the the
files within it are managed by the hackyInstaller application. For example, the
HackyInstaller application manages a file within it called "sensor.properties" that
contains information such as the URL of the hackystat server, the user key associated
with this user, and which sensors are enabled.
(c) Server-side workspace and properties setup. Hackystat does not assume that all
sensor data is associated with a single activity. For these and other reasons, it is
normally helpful to set up "workspace roots" and "project" definitions in a user's
accounts to help organize their sensor data into meaningful collections.
It turns out that, for each of these three kinds of settings, you can choose between
having an "administrator" do it for some or all of the students, or have the students do
it themselves.
In the case of (a), registration, we have a module called hackyApp_Experiment, which
enables a hackystat administrator to enter the email addresses associated with a set of
students, and have the system automatically generate the user keys for each of them.
Alternatively, each student can go to the registration page of the Hackystat server,
enter their email address, and have the user key sent to them via email.
In the case of (b), client-side setup, hackyInstaller has a command line invocation mode
as well as a gui interface. If the hackystat administrator can obtain write access to the
home directories of the students, then the administrator can write a simple shell/batch
script to invoke hackyinstaller appropriately for each student, passing it the student's
home directory and their user key (obtained in (a)). HackyInstaller can then do whatever
client-side setup is necessary.
In the case of (c), the Hackystat Administrator has the ability to login to any user's
account, enabling him/her to do whatever server-side workspace and project setup are
required. Alternatively, the students can do it, although that would require them to
learn more about Hackystat than you might want.
Q3) Is there an easy way to quickly enroll the 60 or so students into the HackyStat
server?
Yes. That's provided by the hackyApp_Experiment module.
2) Student's don't always edit files in the same location.
That shouldn't be a problem.
Q4) Is there an easy way to tell hackyStat that all files from some user/s will
always
be part of the same project?
Well, yes and no. You could simply define a project to contain all known workspaces,
which would be relatively "easy", and then all files will be part of that project. The
problem is that this approach might not be able to aggregate information about activities
on the "same" file.
For example, assume that a student worked on the file Foo.java in the lab, and this file
was located on d:\projects\assignmentA\Foo.java. Then, the student goes home and does
some more work on Foo.java, but this time stores the data in c:\cs201\assA\Foo.java (or
even /usr/local/fred/cs201/assA/Foo.java). What you would normally like to be able to do
is to have the Hackystat server side analyses understand that
d:\projects\assignmentA\Foo.java and c:\cs201\assA\Foo.java and
/usr/local/fred/cs201/assA/Foo.java all refer to the same file. If that's what you need,
then the "easy" way won't work--you have to basically configure each person's account
appropriately to the way they work. (Which isn't really "hard", but the point is that
there's no one-size-fits-all configuration for all 60 students.)
3) Student's don't always work in the lab.
Yes, I've found that myself. Quite annoying. It's like they want to be productive or
something. :-)
Q5) How do you recommend that we ask to the students to install HackyStat at
home, it
should be as simple as possible? Has anybody done any research with students
working at
home?
In my software engineering classes, the students rarely use a lab, and mostly always use
their personal computers, so this is actually the default case, not some exceptional
condition.
Fortunately, this is designed to be pretty easy. In a nutshell, they need to: (a)
download hackyInstaller.jar from your Hackystat server's Help page, (b) invoke it, (c)
bring up the Eclipse sensor configuration panel, (d) specify the directory containing
their local Eclipse installation, and (e) click "install".
Q6) Is the upload mechanism sufficiently robust enough to deal with dial up
connections
that might be cut off during upload?
There is currently no form of transaction support in Hackystat sensor data transmission,
so unfortunately there is the possibility of data being lost. I've frankly not had any
experience with Hackystat sensor data transmission over dial up. In a high speed
connection, sensor transmission occurs intermittently (every 10 minutes or so) and lasts
only 2-3 seconds when it occurs. So this hasn't been an issue for us.
Q7) Does anybody have an add on application that would remind the students to
connect
to the Internet and post the HackyStat results?
There are a few ways to approach this:
(a) You might not need a formal application to do this. When students are working
locally and not connected, the sensor data is cached. Once they are connected, the next
time they bring up Eclipse (or any other instrumented application), the data will be sent.
(b) If you want such an application, then here are some tips. Basically, if there is
local data that has not been sent to a server, then there will exist a directory on the
student's computer called <user.home>/.hackystat/offline and it will contain files. So
it's very easy to determine if data has not been sent. What you could do is write a 50
line Java program that checks for the presence of files in that directory and pops up a
window (or whatever) if it finds something in there. If you want it to run
automatically, then that has to be set up in an OS-specific manner. And, of course, you
have to teach the students how to download it and install it.
4) We have to do an ethics review...
Q8) As part of university policy we have to carry out an ethics review of how
we will
be collecting data, and following approval notify the students and offer an opt
out. I
wonder if anybody else has been through a similar process and has any tips or
comments,
particularly in terms of encouraging the students to participate in the
research.
Yes, we go through that as well. Basically, we guarantee the students that their data
will be kept private and no identifying information will be revealed in any use of their
data in research.
In terms of incentives, our approach is to basically incorporate metrics collection and
analysis into the curriculum. Hackystat becomes the no-brainer solution to satisfying
those particular course requirements.
What is timeframe for deployment of Hackystat?
Cheers,
Philip
