Re: [Soc-coordination] GSoC begins, some important information
On Fri, Jun 03, 2011 at 12:38:17AM -0500, Rudy Godoy Guillén wrote: Hello Ana, I'm sorry for being late on this. I'll write a report as soon as I can, I've also a pending blog update. quick status: we are doing fine, and this weekend I might get to some milestone. ¿Has leido los emails? No has respondido a nada de lo que he preguntado :) Ana thanks! 2011/6/1 Ana Guerrero a...@debian.org First, little reminder, we are in the second week of GSoC and your first report is expected this weekend, between Friday 2 June and Monday 6 June. On Mon, May 23, 2011 at 10:47:21PM +0200, Ana Guerrero wrote: - Please, send me in a private email how to contact you in case we don't have news from you. At least a phone number (include all the needed prefixes for international calls!). Nathan, Rudy, I haven't hear from you yet. - If you haven't yet, please add your project proposal to the Debian wiki so everybody interested in the Debian community can know more about your project and maybe even help you with ideas and suggestions. I can see right now: * Automated Multi-Arch Cross-Building and Bootstrapping This was a late submission and it never made into the wiki. Add it please. APT/Dpkg Transaction Ordering for Safety and Performance http://wiki.debian.org/SummerOfCode2011/AptOrderingCodeImprovements/ChristopherBaines DebDelta APT Native Integration http://wiki.debian.org/SummerOfCode2011/AptDebdeltaIntegration/IshanJayawardena * Dpkg Declarative Diversions http://wiki.debian.org/SummerOfCode2011/DeclarativeDiversions/SamDunne *it this up-todate?* * Backend Tools and Infrastructure for DEX http://wiki.debian.org/SummerOfCode2011/DEX/NathanHandler *this project was asked to concrete more the deliverables in melange and it was never updated. Update please :)* Jigsaw Modularized Java in Debian http://wiki.debian.org/SummerOfCode2011/Jigsaw/GuillaumeMazoyer Python Multi-Build for Python Extensions Packaging http://wiki.debian.org/SummerOfCode2011/PythonMultibuild/mesutcankurt Debian Teams Activity Metrics http://wiki.debian.org/SummerOfCode2011/TeamFeatures/SukhbirS * Compute Clusters Integration for Debian Development and Building http://wiki.debian.org/SummerOfCode2011/BuildWithEucalyptus/RudyGodoy *it this up-todate?* If you project name starts with a * please, update/add it! Ana -- Rudy Godoy Computer Science Student San Pablo Catholic University, Arequipa-Peru (http://www.ucsp.edu.pe) http://htu.pe - Internet Business Software ___ Soc-coordination mailing list Soc-coordination@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
Re: [Soc-coordination] GSoC begins, some important information
And... the previous emails was meant only to Rudy =) Ana ___ Soc-coordination mailing list Soc-coordination@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
[Soc-coordination] Debdelta APT Integration - Report 1
Hi,
My project is integrating debdelta natively to APT. I added my project
proposal to the Debian wiki [1], and there is also a wiki page created
by my mentor in the Ubuntu wiki[2], Michael Vogt, which describes the
design and the implementation plan in detail.
I spent the community boding period to learn the background knowledge
needed for the project. I have been updating Michael regularly and
he's guiding me through my work. First, I refreshed my knowledge about
debdelta by reading its new documentation[3]. Then, I started studying
the design and the internals of APT by reading and debugging its code.
My goal was to find out the functionality of apt when it executes a
download/update process and the classes/functions responsible for it
so that I can do the initial integration and test it. I concentrated
on the fetcher and the line based communication protocol from APT to
the helper methods such as http, file etc. Then I added the new
method/debdelta method and tested it with a simple shell script. The
shell script runs the debdelta method with a .deb and a .debdelta as
the input and verify the resulting deb. To get an idea about how to
write the debdelta method, I studied the existing rred implementation.
This work is related to the first stage mentioned in [2], and
currently I am writing the code to move the resulting deb after
patching from /var/cache/apt/archives/partial to
/var/cache/apt/archives.
Before I started the development, I set up my development environment.
I configured a separate directory as the root for my development
version of APT and created a local repository with sample .deb and
.debdelta files and added an entry for it in the sources.list file to
make APT work in a sandbox environment to keep my original system
intact. I looked into the test/integration/framework script to learn
how to set up a local archive for testing. My APT branch is available
at [4] and I am planning to update it each time I complete a
considerable milestone in the development.
My next plan is to write the required subclasses for pkgAcquire class
that can fetch/handle debdelta files and debdelta index files and
adding suitable functions(such as custom600Headers()) to them to work
with pkgAcquire::Worker::QueueItem() function. The classes that I am
planning to add are,
class pkgAcqDebdeltas : public pkgAcquire::Item {} for
fetching/handling the debdelta files.
class pkgAcqDebdeltaIndex : public pkgAcquire::Item {} for handling
the index file of the debdelta files.
[1]
http://wiki.debian.org/SummerOfCode2011/AptDebdeltaIntegration/IshanJayawardena
[2] https://wiki.ubuntu.com/DebdeltaAptIntegration
[3] http://debdelta.debian.net/html/index.html
[4] https://code.launchpad.net/~udeshike/apt/apt-ishan
--
Regards,
Ishan Jayawardena.
___
Soc-coordination mailing list
Soc-coordination@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
[Soc-coordination] Jigsaw - Report 1
The beginning of this Google Summer of Code has been busy and pretty
cool. Not a lot of code was produced but I did learn a lot of things.
During this GSoC I will work on the next generation JDK. This means that
I will (and I already have) compile the JDK. That's pretty awesome but
it requires a lot of power. That's why Tom Marble created me an account
on his 8 CPUs machine to help me.
Since I'm probably going to sign some packages, I created a new GPG key
with a 4096 RSA strength.
My old key was:
pub 1024D/F144A319 2008-10-18
And the new key is:
pub 4096R/EE2BBBC7 2011-05-03
I cross signed my keys using this great tutorial[1]
I read some documentation about Jigsaw[2] and watch Tom's talk at
DebConf 10[3]
I have spent a little more than on week to understand the current
OpenJDK packaging to be able to rebuild it. I have learnt from where do
the sources come from. The current packaging is a little hard to
understand. There are sources that come from IcedTea[4], OpenJDK[5], and
more[6].
After being able to rebuild the OpenJDK 6 package, I have downloaded the
source of the OpenJDK 8 and Jigsaw. I used the Mercurial forest to get
the sources.
For OpenJDK 8:
- hg clone http://hg.openjdk.java.net/jdk8/jdk8 jdk8
- cd jdk8 sh get_sources.sh
For Jigsaw:
- hg clone http://hg.openjdk.java.net/jigsaw/jigsaw jigsaw
- cd jigsaw sh get_sources.sh
I have managed to build the sources of OpenJDK 8 and Jigsaw using the
same steps:
- cd jigsaw
- . jigbuild
- . jdk/make/jdk_generic_profile.sh
- export ALLOW_DOWNLOADS=true
- make sanity
- make all
Jigsaw specific:
- make modules-sanity
- make modules
Each build took several minutes. But they were successful. Tom helped me
a lot to build OpenJDK 8 and Jigsaw by giving me a solution. For
example, after getting the Mercurial forest, there were still missing
source files and the build failed. Tom told me that I needed to export a
environement variable: ALLOW_DOWNLOADS=true.
After building Jigsaw, I was able to see that the version string is not
legal in Debian and that is a real problem.
I have written some simple Hello World examples to compile and run with
Jigsaw. To modularize a program, it needs to contain a module-info.java
files. This file includes the modules needed to run the program and the
main class to run. For a classic hello world, the module-info.java files
looks like:
module test.hello @ 1 {
requires jdk.base @ 7-ea;
class test.hello.Main;
}
Tom gave me some packages to build: jtharness and jtreg. I succesfully
built jtharness using CDBS[7] and its Ant helper but I'll have to
repackage it to use javahelper instead of CDBS. I still don't try to
build jtreg because it requires javahelper to be built.
My next plan is to get jtharness and jtreg packaged with javahelper soon
and understand what modules does what in Jigsaw. Understaning modules
will help me to write some more complex examples. I hope to start
working on Jigsaw a little more soon (when I my internship will end
[next week]).
[1] http://people.debian.org/~anibal/ksp-dc10/ksp-dc10.html
[2] http://openjdk.java.net/projects/jigsaw/
[3] http://penta.debconf.org/dc10_schedule/events/558.en.html
[4] http://icedtea.classpath.org/download/source/
[5] http://download.java.net/openjdk/jdk6/
[6] http://cacaovm.org/
[7] http://roku.info9.net/~guillaume/jtharness/
--
Guillaume Mazoyer
signature.asc
Description: Digital signature
___
Soc-coordination mailing list
Soc-coordination@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
[Soc-coordination] Declarative Diversions - Report 1
The following is the design spec so far for this project. It has to be fleshed out a bit more but most of the basic structure is there. Anyone who cares to give input is more than welcome. A further update will be posted on Monday to my blog, planet debian, the wiki page and to the dpkg mailing list. DECLARATIVE DIVERSIONS == Introduction A diversion is when it is possible to have dpkg not overwrite a file when it reinstalls the package it belongs to, and to have it put the file from the package somewhere else instead. Declarative diversions involves a new control file with a declarative syntax which dpkg will parse and process directly as part of the package unpack and removal phases, eliminating the problems resulting from non-atomic handling of diversions. -- Topics -- There are a number of topics involved in implementing this kind of project * What syntax do we use for the new control file? * What dpkg does with the control file * How do we handle diversions to a non-existant directory? * How do we order unpacking a new package which adds a diversion? * How do we order removing a package which had a diversion? * How do we handle errors? * What happens to dpkg-divert? - Details - Control File Syntax - It will conform to RFC2822 style with the following format: * Divert-From: * Divert-To: * Blank lines and lines beginning with '#' will be comments 'Divert-To' will be optional and if it is ommitted then files being diverted will have their filename changed to 'file.distrib' The above style has it's advantages, one of the main ones being that there is no need to escape whitespace within filenames. --- Details - Control File Handling --- Within control.tar.gz the file should be named 'diversions' This file is then copied to /var/lib/info/$package.diversions This is ensuring we have a copy of the control file just in case it is needed. Any input on this subject would be appreciated. - Details - Handling Diversions to non-existant directories - Diverting files to directories that don't exist can cause a number of problems. If the package does not 'own' the directory it may be left orphaned on removal of the package The package is responsible for ensuring the availability of the target directory in the unpack phase. --- Details - Ordering Requirements --- =Unpacking a new package that adds a diversion *1. Unpack folders *2. Add diversions *3. Unpack files ==Notes== Performing diversions this way avoids trouble with creating folders and accidentally overwrtiting the wrong version of files. If we don't use this method the problems that can occur include creating folders before the package itself would create them which may cause permission errors. Another major problem would be extracting a new version of the file and overwriting the old one. This might cause the wrong version of the file to be diverted and the package to break. =Removing a package which had a diversion *1. Remove files *2. Remove diversions *3. Remove folders ==Notes== This ensures that all files, diversions and folders are removed correctly Details - Error handling Errors in diversions will have to handled with a great deal of care due to the fact that if they are not the package could be broken. This means that a great deal of checks must be done to ensure that all the files can be diverted properly before any actual diverting takes place. If they can't the package installation/update must be stopped and rolled back to avoid the package being installed incorrectly or broken. --- Details - 'dpkg-divert' --- When we impliment the new diversion method we should keep the current dpkg-divert. This allows maintainers to catch up with the new method without breaking their packages. It also allows maintainers to perform some operations that aren't support by the new method. Example Usage #1 The file to be diverted is '/usr/share/foo' It needs to be moved to '/usr/share/bar' The syntax of the control file would be: #Start File Divert-From: /usr/share/foo Divert-To: /usr/share/bar #End File Example Usage #2 In this example the maintainer doesn't want to move the file to any specific folder The syntax of the control file would be: #Start File Divert-From: /usr/share/foo #End File This would divert the file to '/usr/share/foo.distrib' - Footnotes - RFC2822 Guide: * http://www.faqs.org/rfcs/rfc2822.html First Email Thread on Declarative Diversions (First
[Soc-coordination] Computer clusters integration for Debian development - report 01
Hello, this is the official project report for the Debian GSoC admins and everyone. It's been few days since the official coding phase started. Most of the time I've been figuring out the details regarding integration of non-x86 images to the Eucalyptus IaaS and working on the ARM image, as you'll learn later. I'm also in the process of starting to design the set of tools we'll be delivering as part of the project. The project's goal is to be a useful resource for Debian developers and porters. I've joined some porters list in order to learn what are the things they are struggling with and how the project can help to address them. I'll be starting a wiki page with such topics. Bonding period: Bonding period has been more fruitful than I expected. As mentioned in my previous post, Steffen and I got in touch with the Eucalyptus team, since they show their early interest on the project. We are coordinating cooperation between our teams. Besides my mentor and I defined to start working on the ARM image. Coordination went OK. We'll be having meetings every two weeks. Project Status: Resuming Dominique's work I began working on the ARM image. Aurélien's work[2] was a great resource. Currently I have a working and updated Debian testing ARM image using the versatile kernel that can be used under qemu-arm. I'm currently making tests and figuring out the next phase which is cloudify it. Over this weekend I expect to finish on that side and then began on the design and implementation of the tools that will be the project's result. Until now I haven't faced much issues, but the Eucalyptus part since the docs are targeting x86 and amd_64 archs. However I think I can run qemu-arm on top. I'll talk to them about the idea. Future plans: week 3: - Finish tests with the ARM image. - Design the base tool-set, thinking integration with dpkg-buildpackage and qemubuilder[3]. week 4-8: - Write the necessary code. mid-term evaluation. Other bits: Latest days have been quite busy for me, since I'm still have to attend class, my semester ends in five weeks. This week was quite overwhelming with media, my University published an articlehttp://www.ucsp.edu.pe/index.php/noticias/46-notas-de-prensa/1687-alumno-ucsp-desarrollara-proyecto-para-google.html(Spanish) about me and the program. I'm the third student of the CS school participating in GSoC, in the past we had Gentoo Project and KDE students. Later, a local Cable TV program interview me on the same matter, it went fine. I managed to talk about Debian and what we do (we get free Advertising yay!) :) also about Google's Summer of Code program and invited students to participate both in FLOSS projects and the program. Still don't know if they have put it on air, I think it will be next week. Resources: 1- http://wiki.debian.org/Cloud/CreateImage 2- http://www.aurel32.net/info/debian_arm_qemu.php 3- http://wiki.debian.org/qemubuilder -- Rudy Godoy http://stone-head.org ___ Soc-coordination mailing list Soc-coordination@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
[Soc-coordination] Backend Tools and Infrastructure for DEX - Report 1
Hello everyone, Here is my first official Google Summer of Code report. As I still had classes, the first week of coding was rather slow. However, after my first phone call with my mentor, Matt Zimmerman, things started to pick up. I spent most of my time working on a dashboard/portal for the Debian Derivatives Exchange (DEX). We decided that the best approach to this would be to store as little information ourselves as possible. Instead, we will try to pull most of our information from other sources, such as Debian's BTS. The main advantage to this is that package maintainers do not need to change any of their workflows, but they still get to benefit from DEX's work. Currently, I have a python script that gets a list of usertags and bugs for debian-derivati...@lists.debian.org from the BTS. This script will end up getting run by cron. It stores the information it gets in static HTML files. Finally, JQuery and Javascript are used to present the final dashboard to the user. I created a mockup showing how this looks [1]. I also have a functioning website that looks and acts like the mockup, but the code is not publicly available yet (see below). DEX uses Alioth to host its git branches and website. I have spent some time trying to familiarize myself with the recent changes that have taken place on Alioth, as well as trying to figure out the best way to store and execute my script to generate the website. So far, it looks like people are still deciding on the best way to handle having websites that are updated from VCS hooks (due to things being split between vasks and wagner). I will continue to look into this, and might try using an alternate host until things settle down. Next, I plan to work on updating the dashboard to provide a way to edit, store, and display certain extra fields of information. This is necessary for certain pieces of information that are not appropriate for adding to bugs and in a few other cases. I will also be taking some time to talk to a few people who were active contributors to the last DEX project. This will allow me to learn more about their experience, find out what changes they feel should be made to make things run smoother, and get feedback on my current Summer of Code work. Finally, although I have been posting updates on my Summer of Code work to my blog [2][3], Matt thinks that having all of this information in one place will be useful. As a result, I will be creating another wiki page about the project that will contain all of the information people need to know. This page will be updated as the summer progresses. I am also hoping to have my code available in a public VCS by the time of my next progress update so that people can check it out and provide feedback. In addition to the dashboard, I will be working on creating a few other tools to assist with the creation and tracking of new DEX projects. I will also be coming up with a way to generate graphs to visually allow us to track our progress on projects versus time as well as show the work done by people at the individual level. This last task is part of a goal to get more people involved in the DEX project. While we may only be a few weeks into the actual Google Summer of Code, I have already learned a lot of new things. I am looking forward to working more with Matt and other members of the community over the course of the summer. Thanks, Nathan Handler [1] http://nhandler.files.wordpress.com/2011/05/dexdashboard.png [2] http://nhandler.wordpress.com/2011/05/28/google-summer-of-code-update-1/ [3] http://nhandler.wordpress.com/2011/05/31/google-summer-of-code-update-2/ ___ Soc-coordination mailing list Soc-coordination@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/soc-coordination
