[Tutor] installing where does my money go
Hi everyone,
I'm a total newbie giving his best shot at visualising local educational
institution's faculty budget (over 60 million) to stir up some healthy
democratic debate, by trying to adapt the code from
http://wheredoesmymoneygo.org/getting-started/. Its open source, so I'm trying
out how far I'll wind up :).
I'm installing the setup below (see
http://wheredoesmymoneygo.org/getting-started/).
I've got a mac os 10.6.7, python 2.7.1, pip, and the most recent postgres
installation.
I get stuck at;
"Run setup.py to install all required local packages. (Needs more details...)"
bteeuwen@Ben-3 ~/Sites/wdmmg]$./setup.py
from: can't read /var/mail/setuptools
from: can't read /var/mail/wdmmg
./setup.py: line 4: syntax error near unexpected token `newline'
./setup.py: line 4: `setup('
Then I added this to setup.py:
#!/usr/bin/python
This is where I end:
bteeuwen@Ben-3 ~/Sites/wdmmg]$./setup.py
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:266:
UserWarning: Unknown distribution option: 'paster_plugins'
warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
Anyone any advice where to look for answers? Thx!!
Ben
__
sInstallation and Setup
==
Steps to set up WDMMG from scratch. May not be comprehensive, please edit.
--
Install Postgres (or any other sqlalchemy compatible DB) on your machine.
Create a new database called 'production', add a superuser: note down the
username and password.
$ sudo -u postgres createuser -P nameofuser
$ sudo -u postgres createdb --owner nameofuser nameofdb
--
Check out wdmmg from the Mercurial repository.
$ hg clone http://knowledgeforge.net/okfn/wdmmg
[OUTDATED: For info: This will create a "wdmmg" directory with everything in
it. Inside,
you will find:
- "doc": self-explanatory.
- "econdata": source code for some fo our data packages.
- "wdmmg": source code for the data store.
- "wdmmgrdf": version of the data store with an RDF backend (dormant).
- "flash": source code for the dashboard.
Inside "wdmmg/wdmmg" you will find a pretty standard Pylons application. Thex
main part is a Python package called (you guessed it) "wdmmg". So the actual
code is inside "wdmmg/wdmmg".]
--
Run setup.py to install all required local packages. (Needs more details...)
--
Install 'wdmmg' using pip. NB {your-pyenv] here means
'the location of your virtualenv if you have one'::
$ pip -E {your-pyenv} install -r pip-requirements.txt
--
Create a config file called development.ini (this has the config options needed
to run the site locally):
$ paster make-config wdmmg development.ini
--
Edit development.ini with Solr and SQLAlchemy details for your local machine:
solr.url = url of remote Solr script (can be found in .ini files
on servers), or just 'stub' if you prefer
sqlalchemy.url = postgres://username:password@localhost/production
--
Sync the database tables and set up the application:
$ paster setup-app development.ini
--
For data store theme:
Get main wdmmg theme files from the mercurial repository at::
http://bitbucket.org/okfn/wdmmg-wp-theme
And put them at::
wdmmg/public/themes/wdmmg
--
Install some data (not sure if this really belongs here?)
$ chmod a+x install_data.sh
$ ./install_data.sh
--
Run tests, edit anything that is broken::
$ nosetests
--
Load fixtures so you have some sample data::
$ paster fixtures setup
[
Alternatively, if you want the full datasets, run the install_data script
(slow)::
$ chmod a+x install_data.sh
$ ./install_data.sh
]
--
Finally, run the site from development.ini::
$ paster serve --reload development.ini
--
Start hacking!
Trac repository:
http://knowledgeforge.net/okfn/tasks/query?status=assigned&status=new&status=reopened&component=projects.wdmmg&order=priority&col=id&col=summary&col=cc&col=status&col=owner&col=type&col=priority
How to Upgrade Production Service
=
3 dbs/systems:
* data.wheredoesmymoneygo.org - P1
* data.wheredoesmymoneygo.org.2 - P2
* data.staging.wheredoesmymoneygo.org - S1
Suppose P1 = active production, P2 = inactive production
* Shut down write to the main system (Hack way: LimitExcept GET in Apache)
* Dump active production db and load into inactive production db
* Upgrade inactive system and set to use inactive production db
* Use it and test it
* Switch over from P1 to P2
* In apache move wsgi script to point to the other one and reboot apache2
* If this fails just switch back and you are operational again
_
[Tutor] ImportError: No module named wdmmgext.load
Hi,
Thanks Emile van Sebille for answering my last question.
I'm now testing my imported data and I get 21 errors (see attached). The
majority sounds like:
from wdmmgext.load import uganda
ImportError: No module named wdmmgext.load
I've searched the files that use this module. Attached is an example file.
I see 2 more errors;
1) that the file 'ukgov-finances-cra/cra_2009_db.csv' does not exist.
2) SolrException: HTTP code=404, reason=Not Found
Maybe both are due to the wdmmgext.load error, so I'll ignore this for now and
first try to find answer to my first question.
Thanks in advance for the help!
Ben
import json
import os
import pkg_resources
import wdmmg.model as model
from wdmmgext.load import uganda
class TestDepartments(object):
@classmethod
def setup_class(self):
model.repo.delete_all()
model.Session.remove()
self.name = uganda.dataset_name
filepath = 'wdmmg/tests/uganda_sample.xls'
test_file = os.path.abspath(filepath)
uganda.load_files(filename=test_file, commit_every=10)
model.Session.commit()
model.Session.remove()
@classmethod
def teardown_class(self):
model.repo.delete_all()
model.Session.commit()
model.Session.remove()
# check dataset exists
def test_01_dataset(self):
out = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).first()
assert out, out
# Get our keys, and check values exist for them.
def test_02_classification(self):
for key_name in [u'from', u'time', u'uganda_id', u'gou_vote',
u'vote_name', u'project_code', u'project_name',
u'funded_by_donor',
u'funded_by_govt', u'mtef_sector', u'mtef_reference',
u'swg', u'sector_objective', u'peap1_pillar',
u'peap2_objective', u'peap3_area']:
key =
model.Session.query(model.Key).filter_by(name=key_name).first()
assert key, key_name
count = (model.Session.query(model.ClassificationItem)
.join(model.EnumerationValue)
.filter_by(key=key)
).count()
assert count, (key_name, count)
# Check there are some entries and none of them are null
def test_03_entry(self):
dataset_ = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).one()
count = (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
).count()
assert count, 'There are no Entries'
assert not (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
.filter_by(amount=None)
).first(), 'Some Entries have NULL amounts'
# Look for a 'to' field on the first entry in the dataset.
def test_04_entry_to(self):
dataset_ = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).one()
txn = (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
).first()
classif = txn.classification_as_dict()
print classif['to']
assert classif['to'] == 'uganda-society'
EE....EEE
==
ERROR: test suite for
--
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 208, in run
self.setUp()
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 291, in setUp
self.setupContext(ancestor)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 314, in setupContext
try_run(context, names)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/util.py",
line 478, in try_run
return func()
File "/Users/bteeuwen/Sites/wdmmg/wdmmg/tests/functional/test_aggregate.py",
line 8, in setup_class
Fixtures.setup()
File "/Users/bteeuwen/Sites/wdmmg/wdmmg/lib/cli.py", line 101, in setup
from wdmmgext.load import cofog
ImportError: No module named wdmmgext.load
>> begin captured logging <<
pylons.configuration: DEBUG: Initializing configuration, package: 'wdmmg'
pylons.configuration: DEBUG: Pushing process configuration
pylons.configuration: DEBUG: Adding mako engine with alias None and
{'myghty.data_dir': '/Users/bteeuwen/Sites/wdmmg/pylons_data/templates',
'mako.directories': ['/Users/bteeuwen/Sites/wdmmg/wdmmg/templates'],
'myghty.component_root': [{'templates':
'/Users/bteeuwen/Sites/wdmmg/wdmmg/templates'}], 'kid.encoding': 'utf-8',
'kid.assume_encoding': 'utf-8', 'mako.module_directory':
'/Users/bteeuwen/Sites/wdmmg/pylons_data/templates', 'myghty.allow_globa
[Tutor] ImportError: No module named wdmmgext.load
Hi,
I'm trying to install the code from
http://wheredoesmymoneygo.org/getting-started/ on my local machine. I've got a
mac os 10.6.7, python 2.7.1, pip, and the most recent postgres installation.
I'm now testing imported data and I get 21 errors (see attached). The majority
sounds like:
from wdmmgext.load import uganda
ImportError: No module named wdmmgext.load
I've searched the files that use this module. Attached is an example file.
I see 2 more errors;
1) that the file 'ukgov-finances-cra/cra_2009_db.csv' does not exist.
2) SolrException: HTTP code=404, reason=Not Found
Maybe both are due to the wdmmgext.load error, so I'll ignore this for now and
first try to find answer to my first question.
Thanks in advance for the help!
Ben
import json
import os
import pkg_resources
import wdmmg.model as model
from wdmmgext.load import uganda
class TestDepartments(object):
@classmethod
def setup_class(self):
model.repo.delete_all()
model.Session.remove()
self.name = uganda.dataset_name
filepath = 'wdmmg/tests/uganda_sample.xls'
test_file = os.path.abspath(filepath)
uganda.load_files(filename=test_file, commit_every=10)
model.Session.commit()
model.Session.remove()
@classmethod
def teardown_class(self):
model.repo.delete_all()
model.Session.commit()
model.Session.remove()
# check dataset exists
def test_01_dataset(self):
out = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).first()
assert out, out
# Get our keys, and check values exist for them.
def test_02_classification(self):
for key_name in [u'from', u'time', u'uganda_id', u'gou_vote',
u'vote_name', u'project_code', u'project_name',
u'funded_by_donor',
u'funded_by_govt', u'mtef_sector', u'mtef_reference',
u'swg', u'sector_objective', u'peap1_pillar',
u'peap2_objective', u'peap3_area']:
key =
model.Session.query(model.Key).filter_by(name=key_name).first()
assert key, key_name
count = (model.Session.query(model.ClassificationItem)
.join(model.EnumerationValue)
.filter_by(key=key)
).count()
assert count, (key_name, count)
# Check there are some entries and none of them are null
def test_03_entry(self):
dataset_ = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).one()
count = (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
).count()
assert count, 'There are no Entries'
assert not (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
.filter_by(amount=None)
).first(), 'Some Entries have NULL amounts'
# Look for a 'to' field on the first entry in the dataset.
def test_04_entry_to(self):
dataset_ = (model.Session.query(model.Dataset)
.filter_by(name=self.name)
).one()
txn = (model.Session.query(model.Entry)
.filter_by(dataset_=dataset_)
).first()
classif = txn.classification_as_dict()
print classif['to']
assert classif['to'] == 'uganda-society'
EE....EEE
==
ERROR: test suite for
--
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 208, in run
self.setUp()
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 291, in setUp
self.setupContext(ancestor)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/suite.py",
line 314, in setupContext
try_run(context, names)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/util.py",
line 478, in try_run
return func()
File "/Users/bteeuwen/Sites/wdmmg/wdmmg/tests/functional/test_aggregate.py",
line 8, in setup_class
Fixtures.setup()
File "/Users/bteeuwen/Sites/wdmmg/wdmmg/lib/cli.py", line 101, in setup
from wdmmgext.load import cofog
ImportError: No module named wdmmgext.load
>> begin captured logging <<
pylons.configuration: DEBUG: Initializing configuration, package: 'wdmmg'
pylons.configuration: DEBUG: Pushing process configuration
pylons.configuration: DEBUG: Adding mako engine with alias None and
{'myghty.data_dir': '/Users/bteeuwen/Sites/wdmmg/pylons_data/templates',
'mako.directories': ['/Users/bteeuwen/Sites/wdmmg/wdmmg/templates'],
'myghty.component_root': [{'templates':
'/Users/bteeuwen/Sites/wdmmg/wdmmg/templates'}], 'kid.encoding': 'utf-8
