I finally finished a program that updates the wiki sites based on
EXPORT_SYMBOL_GPL and MODULE_LICENSE calls. What it does is searches the
kernel for files that have these calls in them and appends the
appropriate lines to the end of the wiki sites. It also changes the
license if the function license is more specific (in most cases,
unspecified license -> unspecified version of GPL).
Some warnings:
1. This is the first program I've written for the Free Software
community. I'm sure there are rules I'm breaking and expectations I'm
not meeting. If so, let me know what they are!
2. I wrote this to be run on my computer. I figured that eventually this
stuff would be incorporated into kfv.el. If not, I can always run it
again in the future after more directories have been checked. What this
means is that there is some hardcoded stuff (like the location of the
kernel source) and it uses some nonstandard text-processing programs. If
it's important to make this program easy to use for other people, I'm
happy to make it do that, but I figured since it only needs to be run
once, why make the effort.
3. This program appends the relevant code to the bottom of the wiki
page. I am assuming that other people have not added any of these calls
to their pages. If this is a problem, let me know, and I can add
something that checks to see if they have already been added.
I am attaching the code to the program in case anyone wants to look over
it before I run the program. Finally, I have ran it on the drivers/edac
directory and put the output in the wikiSandbox, so they have names urls
like
http://wiki.gnewsense.org/Main/WikiSandbox--Ubuntu-hardy-linux-2-6-24-12-22--drivers--edac--i82443bxgx-edac-c.
Also, I assume there is some easy way to restore the wiki pages just in
case I mess them up. I think I've done enough testing, but I guess I'm a
little paranoid.
Peter Stevenson
// File Name: license_update.cxx
// Author: Peter Stevenson
// Email Address: [EMAIL PROTECTED]
// Description: Program to update KFV wiki with information gleaned from
// MODULE_LICENSE and EXPORT_SYMBOL_GPL calls in the kernel.
// Last Changed: May 7, 2008
#include <algorithm> // Provides find
#include <cassert> // Provides assert
#include <cstdlib> // Provides EXIT_SUCCESS and system
#include <fstream> // Provides ifstream
#include <iostream> // Provides cin and cout
#include <string> // Provides getline and string
#include <vector> // Provides vector
using namespace std;
// Structure that keeps license information relating to a specific kernel file
// Uses string
struct FileInfo
{
bool to_update;
string file_name, function_license, relevant_code, url, wiki_license;
};
// Structure that keeps information about a particular type of license
// Uses vector
struct LicenseInfo
{
string name;
vector<string> relevant_code, to_update, not_to_update;
};
void compare_licenses(FileInfo& record, vector<LicenseInfo>& licenses);
// Precondition: record has been created by get_kernel_data. There must be a
// record in licenses that corresponds to the license in record.
// Postcondition: record.to_update is true if record.function_license is more
// specific than record.wiki_license.
void find_replace(string& str, string to_replace, string replacement);
// Precondition: None.
// Postcondition: All instances of to_replace in str have been changed to
// replacement.
vector<FileInfo> get_kernel_data(const string PATH, const string DIRECTORY,
const string URL);
// Precondition: PATH is the full path name to the kernel, DIRECTORY is the
// name of the directory to be searched, and URL is the prefix for the
// gNewSense wiki KFV site that is to be updated.
// Postcondition: Returns a vector of FileInfos. Each file in the kernel that
// has a MODULE_LICENSE or an EXPORT_SYMBOL_GPL call in it now has a
// corresponding record in the returned vector. The members file_name,
// relevant_code and url are filled with the appropriate data.
void get_wiki_data(FileInfo record, const string FILE_NAME);
// Precondition: record has been produced by get_kernel_data. FILE_NAME is a
// valid name of a file, although the file does not necessarily exist.
// Postcondition: The license information in record.url has been saved in the
// file called FILE_NAME.
void get_url(FileInfo& record, const string DIRECTORY, const string URL);
// Precondition: record.file_name is a correct full path to a file in the
// kernel, DIRECTORY is the directory in the kernel you want to search, and
// URL is the prefix for the gNewSense wiki KFV site that is to be updated.
// Postcondition: record.url is a url to the gNewSense wiki KFV page
// corresponding to record.file_name.
string get_wiki_license(const string FILE_NAME);
// Precondition: FILE_NAME is the name of a text file in the current directory.
// Returns the first line of the file called FILE_NAME.
void identify_license(FileInfo& record, vector<LicenseInfo>& licenses);
// Precondition: record has been produced by get_kernel_data.
// Postcondition: record.function_license is filled with the appropriate license
// based on record.relevant_code. A LicenseInfo record in licenses has either
// been created or updated with data from record.
bool is_only_export_statements(string relevant_code);
// Returns true if every line of relevant code begins with "EXPORT_SYMBOL_GPL".
// Otherwise returns false.
string process_input(char input);
// Returns a proper license name from condensed user input. Uses same code as
// kfv.el.
void update_data(FileInfo& record, const string FILE_NAME,
vector<LicenseInfo>& licenses);
// Precondition: record has been produced by get_kernel_data. FILE_NAME is the
// name of a file that contains the license information from the record.url.
// Postcondition: record.wiki_license has been filled by the license on the KFV
// wiki site. record.function_license has been filled with the appropriate
// license based on record.relevant_code. Finally, if the function license is
// more specific than the wiki license, the wiki license has been replaced by
// the function license.
void update_wiki(FileInfo record, const string FILE_NAME);
// Precondition: record has been produced by get_kernel_data and has had
// update_data run on it. FILE_NAME is the name of a text file in the
// current directory.
// Postcondition: The wiki site record.url has been updated based on the
// information from the kernel file.
// Uses string and vector
int main( )
{
const string DIRECTORY = "drivers",
PATH = "/usr/local/src/linux-2.6.24_12.22/",
URL = "http://wiki.gnewsense.org/Kernel/Ubuntu-hardy-linux-2-6-24-12-22--";,
WIKI_LICENSE_FILE_NAME = "tmp";
vector<FileInfo> data = get_kernel_data(PATH, DIRECTORY, URL);
vector<LicenseInfo> licenses;
vector<FileInfo>::iterator iter;
for (iter = data.begin( ); iter != data.end( ); iter++)
{
(*iter).to_update = false;
get_wiki_data(*iter, WIKI_LICENSE_FILE_NAME);
update_data(*iter, WIKI_LICENSE_FILE_NAME, licenses);
update_wiki(*iter, WIKI_LICENSE_FILE_NAME);
}
return(EXIT_SUCCESS);
}
// Uses string and vector
void compare_licenses(FileInfo& record, vector<LicenseInfo>& licenses)
{
vector<LicenseInfo>::iterator iter;
vector<string>::iterator i2;
string input;
for (iter = licenses.begin( ); iter != licenses.end( ); iter++)
if (record.function_license == (*iter).name)
{
for (i2 = (*iter).to_update.begin( ); i2 != (*iter).to_update.end( );
i2++)
if (record.wiki_license == *i2)
{
record.to_update = true;
return;
}
for (i2 = (*iter).not_to_update.begin( ); i2 !=
(*iter).not_to_update.end( ); i2++)
if (record.wiki_license == *i2)
return;
break;
}
cout << "File name is: " << record.file_name << endl
<< "Function license is: " << record.function_license << endl
<< "Wiki license is: " << record.wiki_license << endl
<< "Modify?";
cin >> input;
if (input == "n")
(*iter).not_to_update.push_back(record.wiki_license);
else
{
(*iter).to_update.push_back(record.wiki_license);
record.to_update = true;
}
}
// Uses string
void find_replace(string& str, string to_replace, string replacement)
{
string::size_type pos = 0;
while ((pos = str.find(to_replace)) != string::npos)
{
str.replace(pos, to_replace.size( ), replacement);
pos++;
}
return;
}
// Uses cassert, cstdlib, fstream, string and vector
vector<FileInfo> get_kernel_data(const string PATH, const string DIRECTORY,
const string URL)
{
vector<FileInfo> data;
const string TMP_FILE_NAME = "tmp", LICENSES_FILE_NAME = "licenses";
string command =
"echo \"MODULE_LICENSE\nEXPORT_SYMBOL_GPL\nDRIVER_LICENSE\" > " +
LICENSES_FILE_NAME, next;
ifstream in_stream;
FileInfo record;
system(command.c_str( ));
command = "fgrep -rHf " + LICENSES_FILE_NAME + " " + PATH + DIRECTORY + " > "
+ TMP_FILE_NAME;
system(command.c_str( ));
in_stream.open(TMP_FILE_NAME.c_str( ));
assert(!in_stream.fail( ));
while (getline(in_stream, next, ':'))
{
if (data.size( ) == 0)
{
record.file_name = next;
getline(in_stream, record.relevant_code);
get_url(record, DIRECTORY, URL);
data.push_back(record);
}
else if (next != data.back( ).file_name)
{
record.file_name = next;
getline(in_stream, record.relevant_code);
get_url(record, DIRECTORY, URL);
data.push_back(record);
}
else
{
getline(in_stream, next);
data.back( ).relevant_code.append("\n" + next);
}
}
in_stream.close( );
command = "rm -f " + TMP_FILE_NAME + " " + LICENSES_FILE_NAME;
system(command.c_str( ));
return(data);
}
// Uses algorithm, iostream, string and vector
string get_license(string relevant_code, vector<LicenseInfo>& licenses)
{
}
// Uses string
void get_url(FileInfo& record, const string DIRECTORY, const string URL)
{
string file2url;
string::size_type pos;
file2url = record.file_name;
pos = file2url.find(DIRECTORY);
file2url.erase(0, pos);
find_replace(file2url, "/", "--");
find_replace(file2url, ".", "-");
find_replace(file2url, "_", "-");
file2url = URL + file2url;
record.url = file2url;
return;
}
// Uses cstdlib and string
void get_wiki_data(FileInfo record, const string FILE_NAME)
{
string command = "wget -qO \"/dev/null\" --no-proxy --keep-session-cookies --save-cookies=new-cookies.txt --post-data='authid=petercstevenson&authpw=pet4spik' 'http://wiki.gnewsense.org/Main/HomePage?action=login' && mv new-cookies.txt cookies.txt && wget --no-proxy --load-cookies=cookies.txt -qO- '" + record.url + "?action=edit' | tr \"\\n\" \"\\024\" | grep -o \"<textarea.*<\\/textarea>\" | sed 's/<[^>]*>//g' | tr \"\\024\" \"\\n\" | unhtml | ascii2uni -qa D > tmp";
system(command.c_str( ));
}
// Uses cassert, fstream and string
string get_wiki_license(const string FILE_NAME)
{
string license;
ifstream in_stream;
in_stream.open(FILE_NAME.c_str( ));
assert(!in_stream.fail( ));
getline(in_stream, license);
in_stream.close( );
return(license);
}
// Uses iostream, string and vector
void identify_license(FileInfo& record, vector<LicenseInfo>& licenses)
{
bool is_new = true;
char input;
vector<LicenseInfo>::iterator iter;
if (is_only_export_statements(record.relevant_code))
record.function_license = "GPL (no version specified)";
else
{
for (iter = licenses.begin( ); iter != licenses.end( ); iter++)
if (find((*iter).relevant_code.begin( ), (*iter).relevant_code.end( ),
record.relevant_code) != (*iter).relevant_code.end( ))
{
record.function_license = (*iter).name;
return;
}
do
{
cout << "\nRelevant code is:\n" << record.relevant_code << endl
<< "Please type in the license code or '?' for help: ";
cin >> input;
if (input == '?')
cout << "0: GPLv3\n"
<< "1: GPLv2 or later\n"
<< "2: GPLv2\n"
<< "3: GPLv1\n"
<< "4: LGPLv3\n"
<< "5: LGPLv2.1\n"
<< "6: LGPLv2.0\n"
<< "7: GFDL v1.2\n"
<< "8: GFDL v1.1\n"
<< "9: Modified BSD license (3-clause)\n"
<< "a: FreeBSD license (2-clause)\n"
<< "b: OpenIB.org BSD license\n"
<< "c: GPLv2 / Modified BSD license\n"
<< "d: GPLv2 / FreeBSD license\n"
<< "e: GPLv2 w/in kernel; otherwise Modified BSD license\n"
<< "f: GPLv2 / MPL v1.1\n"
<< "g: GPL / FreeBSD license\n"
<< "h: X11 (aka MIT) License\n"
<< "i: CPL 1.0 / Modified BSD license / GPLv2\n"
<< "j: GPLv2 + FreeBSD license w/in kernel; otherwise FreeBSD license\n"
<< "k: Public Domain\n"
<< "l: No license, so assumed to be GPLv2.\n"
<< "m: GPL (no version specified)\n"
<< "n: Other software license (free)\n"
<< "o: REPLACE WITH CUSTOM LICENSE TEXT\n"
<< "p: Other software license (non-free)\n";
} while (input == '?');
record.function_license = process_input(input);
}
for (iter = licenses.begin( ); iter != licenses.end( ); iter++)
if ((*iter).name == record.function_license)
{
(*iter).relevant_code.push_back(record.relevant_code);
return;
}
vector<string> new_code;
LicenseInfo new_record;
new_code.push_back(record.relevant_code);
new_record.name = record.function_license;
new_record.relevant_code = new_code;
licenses.push_back(new_record);
}
// Uses string
bool is_only_export_statements(string relevant_code)
{
string::size_type pos = 0;
bool is_all_export = true;
do
{
if ((relevant_code.size( ) - pos) > 17)
if (relevant_code.substr(pos, 17) != "EXPORT_SYMBOL_GPL")
is_all_export = false;
pos = relevant_code.find("\n", pos);
} while (pos++ < relevant_code.size( ));
return(is_all_export);
}
// Uses string
string process_input(char input)
{
switch(input)
{
case '0':
return("GPLv3");
case '1':
return("GPLv2 or later");
case '2':
return("GPLv2");
case '3':
return("GPLv1");
case '4':
return("LGPLv3");
case '5':
return("LGPLv2.1");
case '6':
return("LGPLv2.0");
case '7':
return("GFDL v1.2");
case '8':
return("GFDL v1.1");
case '9':
return("Modified BSD license (3-clause)");
case 'a':
return("FreeBSD license (2-clause)");
case 'b':
return("OpenIB.org BSD license");
case 'c':
return("GPLv2 / Modified BSD license");
case 'd':
return("GPLv2 / FreeBSD license");
case 'e':
return("GPLv2 w/in kernel; otherwise Modified BSD license");
case 'f':
return("GPLv2 / MPL v1.1");
case 'g':
return("GPL / FreeBSD license");
case 'h':
return("X11 (aka MIT) License");
case 'i':
return("CPL 1.0 / Modified BSD license / GPLv2");
case 'j':
return("GPLv2 + FreeBSD license w/in kernel; otherwise FreeBSD license");
case 'k':
return("Public Domain");
case 'l':
return("No license, so assumed to be GPLv2.");
case 'm':
return("GPL (no version specified)");
case 'n':
return("Other software license (free)");
case 'o':
return("REPLACE WITH CUSTOM LICENSE TEXT");
case 'p':
return("Other software license (non-free)");
default:
return("REPLACE WITH CUSTOM LICENSE TEXT");
}
}
// Uses string and vector
void update_data(FileInfo& record, const string FILE_NAME,
vector<LicenseInfo>& licenses)
{
record.wiki_license = get_wiki_license(FILE_NAME);
identify_license(record, licenses);
compare_licenses(record, licenses);
}
// Uses cstdlib and string
void update_wiki(FileInfo record, const string FILE_NAME)
{
string command, license;
string::size_type pos = 0;
if (record.to_update)
{
command = "sed -i \"1c \\ " + record.function_license + "\" " + FILE_NAME;
system(command.c_str( ));
license = record.function_license;
}
else
license = record.wiki_license;
do
{
record.relevant_code.insert(pos, " ");
pos = record.relevant_code.find("\n", pos);
} while (pos++ < record.relevant_code.size( ));
command = "echo \"" + record.relevant_code + "\" >> " + FILE_NAME;
system(command.c_str( ));
command = "echo \"author=petercstevenson&csum=" + record.wiki_license +
"&post= Save &text= \" > post-file";
system(command.c_str( ));
command = "cat " + FILE_NAME + " >> post-file";
system(command.c_str( ));
command = "wget -qO \"/dev/null\" --no-proxy --load-cookies=cookies.txt --post-file='./post-file' '" + record.url + "?action=edit'";
system(command.c_str( ));
}
_______________________________________________
gNewSense-users mailing list
[email protected]
http://lists.nongnu.org/mailman/listinfo/gnewsense-users
