I want to make a bit of an announcement for people who don't browse the Github issues list. I've been working on setting up continuous integration, a la Travis, but for Windows using AppVeyor - http://blog.appveyor.com/2014/02/19/appveyor-20-dedicated-build-vms-parallel-testing-nuget-deployment. I have a WIP PR for Julia itself here https://github.com/JuliaLang/julia/pull/6028 with some discussion, but there are some remaining problems I have to solve to get it working for the main Julia codebase.
What does work now, and I encourage package maintainers who've been looking for cross-platform testing to set up, is using AppVeyor for Julia packages. Like Travis, it's free for open source projects, but currently only allows one concurrent build at a time on the free plan, time limit 30 minutes per build. I have a very simple example configuration file set up here https://github.com/tkelman/JSON.jl/blob/master/appveyor.yml and you can see what the build result looks like here https://ci-beta.appveyor.com/project/tkelman/json-jl/build/1.0.31. Not much to look at since JSON.jl is a simple self-contained package, but you can see how similar the appveyor.yml configuration file is to the existing travis.yml. I'm using the Julia binary installer instead of the apt-get PPA's, and creating a symlink in Windows requires admin rights so I had to change `ln -s` to `cp -r` (there may be a way to elevate permissions in AppVeyor but I'm not sure how). I think this can be a valuable service to identify Windows bugs faster, since the number of Julia developers who use Windows on a regular basis is understandably limited. AppVeyor's developer/founder has been in touch and would be excited to see more users of the tool he's put together, and he's been very responsive to troubleshooting and feature requests. -Tony