[PATCH v3] git-remote-mediawiki: bugfix for pages w/ >500 revisions
Mediawiki introduces a new API for queries w/ more than 500 results in version 1.21. That change triggered an infinite loop while cloning a mediawiki with such a page. The latest API renamed and moved the "continuing" information in the response, necessary to build the next query. The code failed to retrieve that information but still detected that it was in a "continuing query". As a result, it launched the same query over and over again. If a "continuing" information is detected in the response (old or new), the next query is updated accordingly. If not, we quit assuming it's not a continuing query. Signed-off-by: Benoit Person Reported-by: Benjamin Cathey --- contrib/mw-to-git/git-remote-mediawiki.perl | 14 -- contrib/mw-to-git/t/t9365-continuing-queries.sh | 23 +++ 2 files changed, 35 insertions(+), 2 deletions(-) create mode 100755 contrib/mw-to-git/t/t9365-continuing-queries.sh diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index c9a4805..f1db5d2 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -625,6 +625,9 @@ sub fetch_mw_revisions_for_page { rvstartid => $fetch_from, rvlimit => 500, pageids => $id, + + # let the mediawiki know that we support the latest API + continue => '', }; my $revnum = 0; @@ -640,8 +643,15 @@ sub fetch_mw_revisions_for_page { push(@page_revs, $page_rev_ids); $revnum++; } - last if (!$result->{'query-continue'}); - $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + + if ($result->{'query-continue'}) { # For legacy APIs + $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + } elsif ($result->{continue}) { # For newer APIs + $query->{rvstartid} = $result->{continue}->{rvcontinue}; + $query->{continue} = $result->{continue}->{continue}; + } else { + last; + } } if ($shallow_import && @page_revs) { print {*STDERR} " Found 1 revision (shallow import).\n"; diff --git a/contrib/mw-to-git/t/t9365-continuing-queries.sh b/contrib/mw-to-git/t/t9365-continuing-queries.sh new file mode 100755 index 000..27e267f --- /dev/null +++ b/contrib/mw-to-git/t/t9365-continuing-queries.sh @@ -0,0 +1,23 @@ +#!/bin/sh + +test_description='Test the Git Mediawiki remote helper: queries w/ more than 500 results' + +. ./test-gitmw-lib.sh +. $TEST_DIRECTORY/test-lib.sh + +test_check_precond + +test_expect_success 'creating page w/ >500 revisions' ' + wiki_reset && + for i in `test_seq 501` + do + echo "creating revision $i" && + wiki_editpage foo "revision $i" true + done +' + +test_expect_success 'cloning page w/ >500 revisions' ' + git clone mediawiki::'"$WIKI_URL"' mw_dir +' + +test_done -- 1.8.4.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] git-remote-mediawiki: bugfix for pages w/ >500 revisions
On 23 September 2013 19:58, Matthieu Moy wrote: > I'd rather have the comments say "# API version < X" and "# API version >>= X". Next time the API change, "new" Vs "old" will become meaningless. done, thanks On 23 September 2013 20:26, Jonathan Nieder wrote: > Some distros (e.g., Debian) occasionally do run the testsuite > automatically, but it is still fine since they have a timeout that > varies by platform to detect if the test has stalled. I suppose > ideally git's test harness could learn to do the same thing some day, > but for now it's easier one level above since an appropriate timeout > depends on the speed on the platform, what else is creating load on > the test machine, and other factors that are probably not easy for us > to guess. great explanation, thanks On 23 September 2013 22:17, Eric Sunshine wrote: > s/seq/test_seq/ done, thanks > d17cf5f3a32f07bf (tests: Introduce test_seq; 2012-08-03) > >> + do >> + echo "creating revision $i" > > Do you want to end this line with '&&'? The way it's intended is that it's more a debug information to see how it's going on (creating >500 revs is *quite* long). If I understand it correctly, using '&&' would mean that the return value of the echo statement will be tested for success ? Anyway, I am not sure it makes sense to fail on a "debug echo" ? -- Benoit Person -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] git-remote-mediawiki: bugfix for pages w/ >500 revisions
Mediawiki introduces a new API for queries w/ more than 500 results in version 1.21. That change triggered an infinite loop while cloning a mediawiki with such a page. The latest API renamed and moved the "continuing" information in the response, necessary to build the next query. The code failed to retrieve that information but still detected that it was in a "continuing query". As a result, it launched the same query over and over again. If a "continuing" information is detected in the response (old or new), the next query is updated accordingly. If not, we quit assuming it's not a continuing query. Signed-off-by: Benoit Person Reported-by: Benjamin Cathey --- contrib/mw-to-git/git-remote-mediawiki.perl | 14 -- contrib/mw-to-git/t/t9365-continuing-queries.sh | 24 2 files changed, 36 insertions(+), 2 deletions(-) create mode 100755 contrib/mw-to-git/t/t9365-continuing-queries.sh diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index c9a4805..f1db5d2 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -625,6 +625,9 @@ sub fetch_mw_revisions_for_page { rvstartid => $fetch_from, rvlimit => 500, pageids => $id, + + # let the mediawiki know that we support the latest API + continue => '', }; my $revnum = 0; @@ -640,8 +643,15 @@ sub fetch_mw_revisions_for_page { push(@page_revs, $page_rev_ids); $revnum++; } - last if (!$result->{'query-continue'}); - $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + + if ($result->{'query-continue'}) { # For legacy APIs + $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + } elsif ($result->{continue}) { # For newer APIs + $query->{rvstartid} = $result->{continue}->{rvcontinue}; + $query->{continue} = $result->{continue}->{continue}; + } else { + last; + } } if ($shallow_import && @page_revs) { print {*STDERR} " Found 1 revision (shallow import).\n"; diff --git a/contrib/mw-to-git/t/t9365-continuing-queries.sh b/contrib/mw-to-git/t/t9365-continuing-queries.sh new file mode 100755 index 000..6fb5df4 --- /dev/null +++ b/contrib/mw-to-git/t/t9365-continuing-queries.sh @@ -0,0 +1,24 @@ +#!/bin/sh + +test_description='Test the Git Mediawiki remote helper: queries w/ more than 500 results' + +. ./test-gitmw-lib.sh +. ./push-pull-tests.sh +. $TEST_DIRECTORY/test-lib.sh + +test_check_precond + +test_expect_success 'creating page w/ >500 revisions' ' + wiki_reset && + for i in $(seq 1 501) + do + echo "creating revision $i" + wiki_editpage foo "revision $i" true + done +' + +test_expect_success 'cloning page w/ >500 revisions' ' + git clone mediawiki::'"$WIKI_URL"' mw_dir +' + +test_done -- 1.8.4.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-remote-mediawiki: bugfix for pages w/ >500 revisions
Mediawiki introduced a new API for queries w/ more than 500 results in version 1.21. That change triggered an infinite loop while cloning a mediawiki with such a page. Fix that while still preserving the old behavior for old APIs. Signed-off-by: Benoit Person Reported-by: Benjamin Cathey --- Patch tested for all mediawiki versions from 1.19 to 1.21. For now, if the tests suite is run without the fix, the new test introduces an infinite loop. I am not sure if this should be handled ? (a timeout of some kind maybe ?) contrib/mw-to-git/git-remote-mediawiki.perl | 14 -- contrib/mw-to-git/t/t9365-continuing-queries.sh | 24 2 files changed, 36 insertions(+), 2 deletions(-) create mode 100755 contrib/mw-to-git/t/t9365-continuing-queries.sh diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index c9a4805..2d7af57 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -625,6 +625,9 @@ sub fetch_mw_revisions_for_page { rvstartid => $fetch_from, rvlimit => 500, pageids => $id, + +# let the mediawiki knows that we support the latest API +continue => '', }; my $revnum = 0; @@ -640,8 +643,15 @@ sub fetch_mw_revisions_for_page { push(@page_revs, $page_rev_ids); $revnum++; } - last if (!$result->{'query-continue'}); - $query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; + +if ($result->{'query-continue'}) { # For legacy APIs +$query->{rvstartid} = $result->{'query-continue'}->{revisions}->{rvstartid}; +} elsif ($result->{continue}) { # For newer APIs +$query->{rvstartid} = $result->{continue}->{rvcontinue}; +$query->{continue} = $result->{continue}->{continue}; +} else { +last; +} } if ($shallow_import && @page_revs) { print {*STDERR} " Found 1 revision (shallow import).\n"; diff --git a/contrib/mw-to-git/t/t9365-continuing-queries.sh b/contrib/mw-to-git/t/t9365-continuing-queries.sh new file mode 100755 index 000..6fb5df4 --- /dev/null +++ b/contrib/mw-to-git/t/t9365-continuing-queries.sh @@ -0,0 +1,24 @@ +#!/bin/sh + +test_description='Test the Git Mediawiki remote helper: queries w/ more than 500 results' + +. ./test-gitmw-lib.sh +. ./push-pull-tests.sh +. $TEST_DIRECTORY/test-lib.sh + +test_check_precond + +test_expect_success 'creating page w/ >500 revisions' ' + wiki_reset && + for i in $(seq 1 501) + do + echo "creating revision $i" + wiki_editpage foo "revision $i" true + done +' + +test_expect_success 'cloning page w/ >500 revisions' ' + git clone mediawiki::'"$WIKI_URL"' mw_dir +' + +test_done -- 1.8.4.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v8 5/7] git-remote-mediawiki: Factoring code between git-remote-mediawiki and Git::Mediawiki
From: Benoit Person For now, Git::Mediawiki contains nothing. This first patch moves some of git-remote-mediawiki.perl's factorisable code into Git::Mediawiki. In the same time, it removes the side effects of that code and renames the fucntions and constants moved to expose a better API. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 805f42a..47fe4f4 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 71baf8a..e40c034 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use consta
[PATCH v8 7/7] git-remote-mediawiki: Add preview subcommand into git mw
From: Benoit Person In the current state, a user of git-remote-mediawiki can edit the markup text locally, but has to push to the remote wiki to see how the page is rendererd. Add a new 'git mw preview' command that allows rendering the markup text on the remote wiki without actually pushing any change on the wiki. This uses Mediawiki's API to render the markup and inserts it in an actual HTML page from the wiki so that CSS can be rendered properly. Most links should work when the page exists on the remote. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 310 - 2 files changed, 311 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 47fe4f4..d13c4df 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 4a3e4a9..28df3ee 100755 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,6 +12,14 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); # By default, use UTF-8 to communicate with Git and the user binmode STDERR, ':encoding(UTF-8)'; @@ -26,9 +34,26 @@ sub v_print { return; } +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] ); # Search for sub-command @@ -47,6 +72,288 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's +upstream if --remote is not set. If that remote is not found or if it +is not a mediawiki, it lists all mediawiki remotes configured and asks +you to replay your command with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in +the current dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed +by the remote mediawiki and combined with a template retrieved from +the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it +in your default web browser if the option --autoload is present. + +OPTIONS: +-r , --remote +If the remote is a mediawiki, the template and the parse engine +used for the preview will be those of that remote. +If not, a list of valid remotes will be shown. + +-a, --autoload +Try to load the HTML output in a new tab (or new window) of your +default web browser. + +-o , --output +Change the HTML output filename. Default filename is based on the +input filename with its extension replaced by '.html'. + +-v, --verbose +Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($new_content, $template); + my $file_content; + + i
[PATCH v8 4/7] git-remote-mediawiki: Update tests to run with the new bin-wrapper
From: Benoit Person Until now, if git-remote-mediawiki was not installed, the test suite copied it to the toplevel directory. This solution pollutes the directory with untracked files. Plus, we would need to copy the new git-mw.perl file to test it too. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/t/test-gitmw-lib.sh | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh index bb76cee..ca6860f 100755 --- a/contrib/mw-to-git/t/test-gitmw-lib.sh +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh @@ -62,12 +62,8 @@ test_check_precond () { test_done fi - if [ ! -f "$GIT_BUILD_DIR"/git-remote-mediawiki ]; - then - echo "No remote mediawiki for git found. Copying it in git" - echo "cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/" - ln -s "$GIT_BUILD_DIR"/contrib/mw-to-git/git-remote-mediawiki "$GIT_BUILD_DIR" - fi + GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd "../.." && pwd) + PATH="$GIT_EXEC_PATH"'/bin-wrapper:'"$PATH" if [ ! -d "$WIKI_DIR_INST/$WIKI_DIR_NAME" ]; then -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v8 1/7] git-remote-mediawiki: Introduction of Git::Mediawiki.pm
From: Benoit Person We would want to allow the user to preview what he has edited locally before pushing it out (and thus creating a non-removable revision in the mediawiki's history). This patch introduces a new perl package in which we will be able to share code between that new tool and the remote helper: git-remote-mediawiki.perl. A perl package offers the best way to handle such case: Each script can select what should be imported in its namespace. The package namespacing limits the use of side effects in the shared code. An alternate solution is to concatenate a "toolset" file with each *.perl when 'make'-ing the project. In that scheme, everything is imported in the script's namespace. Plus, files should be renamed in order to chain to Git's toplevel makefile. Hence, this solution is not acceptable. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 24 contrib/mw-to-git/Makefile | 24 +--- 2 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 000..805f42a --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,24 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..a6f8b24 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,18 +2,36 @@ # Copyright (C) 2013 # Matthieu Moy # -## Build git-remote-mediawiki +# To install, run Git's toplevel 'make install' then run: +# +# make install +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +install_pm: + install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + perlcritic: perlcritic -2 *.perl -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v8 6/7] git-remote-mediawiki: Adding git-mw command
From: Benoit Person For now, git-remote-mediawiki is only a remote-helper. This patch adds a new toolset script in which we will be able to build new tools for git-remote-mediawiki. This toolset uses a subcommand-mechanism to launch the proper action. For now only the 'help' subcommand is implemented. It also provides some generic code for the verbose and help command line options. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 ++--- contrib/mw-to-git/git-mw.perl | 60 +++ 2 files changed, 64 insertions(+), 3 deletions(-) create mode 100755 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index c5e66df..775cb07 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -14,6 +14,7 @@ GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -27,15 +28,15 @@ install_pm: install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100755 index 000..4a3e4a9 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v8 3/7] git-remote-mediawiki: New git bin-wrapper for developement
From: Benoit Person The introduction of the Git::Mediawiki package makes it impossible to test, without installation, git-remote-mediawiki and git-mw. Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the use of the developement version of the Git::Mediawiki package, bypassing its installed version if any. An alternate solution was to 'install' all the files required at each build but it pollutes the toplevel with untracked files. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 6 ++ contrib/mw-to-git/bin-wrapper/git | 14 ++ 2 files changed, 20 insertions(+) create mode 100755 contrib/mw-to-git/bin-wrapper/git diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index a6f8b24..c5e66df 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,6 +2,12 @@ # Copyright (C) 2013 # Matthieu Moy # +# To build and test: +# +# make: +# bin-wrapper/git mw preview Some_page.mw +# bin-wrapper/git clone mediawiki::http://example.com/wiki/ +# # To install, run Git's toplevel 'make install' then run: # # make install diff --git a/contrib/mw-to-git/bin-wrapper/git b/contrib/mw-to-git/bin-wrapper/git new file mode 100755 index 000..59c1420 --- /dev/null +++ b/contrib/mw-to-git/bin-wrapper/git @@ -0,0 +1,14 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +GIT_ROOT_DIR=../../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"${GITPERLLIB:+:$GITPERLLIB}" +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$PATH" + +export GITPERLLIB PATH + +exec "${GIT_EXEC_PATH}/bin-wrappers/git" "$@" -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v8 0/7] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from v7: - Update commit message of 1/7: remove the obvious (we need a way to share code between scripts) and make the commit message less vague. - Update commit message of 2/7: it's useless to do the same update on GIT_TEMPLATE_DIR and GIT_TEXTDOMAINDIR since the codes using them do not seem to handle multiple dirs. - Remove trailing ':' while concatening GITPERLLIB to itself in files: - wrap-for-bin.sh - contrib/mw-to-git/bin-wrapper/git changes from v6: - Rewrite of bin-wrapper/git in 3/7 - Update wrap-for-bin.sh and make bin-wrappers chainable it removes the need for a GPLEXTEA since we only have to first define GITPERLLIB, the bin-wrappers will no longer overwrite it. (2/7) Maybe we should apply the same mechanism to all variables in wrap-for-bin.sh ? (there is no need for it right now though) - Replace the symlink created in the test suite (if git-remote-mediawiki was not installed) by the new bin-wrapper introduced in 3/4 (4/7). - Small typos in the commit messages changes from v5: - Move git bin-wrapper 'git' into bin-wrapper/ - Updates its GIT_ROOT_DIR accordingly - Updates the Makefile to use that new path in the comments changes from v4: - Rebase on latest master - Typos in commits messages and code - Comments in Makefile - Factoring the conversion from relatives links to absolute ones in `git mw preview` - Updating that "conversion" mechanism to not convert links with an anchor '#'. - git-mw should be executable now. changes from v3: - Rewrite all commit messages. - No more "\ No newline at end of file". - Rename GitMediawiki.pm into Git::Mediawiki.pm (so it moves GitMedawiki.pm into Git/Mediawiki.pm). - Remove from the Makefile the copy_pm target (see below 'Add a bin-wrapper'). - Use of 'install' insted of 'cp' in the Makefile. - Comment on the install_pm target in the Makefile. - Add a bin-wrapper for git to test scripts without 'make install'-ing them. - Move verbose option handling from previous v3-4/4 (introduction of preview tool) into v4-4/5 (introduction of git-mw). - Refactor some code into subroutines to clean the global 'preview' subroutine. - Rewrite some error messages to make them more concise while still giving the same amount of information. - Use 'remote.${remote_name}.mwIDcontent' instead of 'mediawiki.IDContent' as config item for the lookup ID used to combine template + new content. - Remove comments about what's going on in the preview subroutine. - Use 'clean_filename' (and not 'smudge_filename') in the preview tool to find the correct mediawiki page name based on a filename. - Remove space/tab mixup in the 'help' subroutine. changes from v2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from v1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (7): git-remote-mediawiki: Introduction of Git::Mediawiki.pm wrap-for-bin: Make bin-wrappers chainable git-remote-mediawiki: New git bin-wrapper for developement git-remote-mediawiki: Update tests to run with the new bin-wr
[PATCH v8 2/7] wrap-for-bin: Make bin-wrappers chainable
From: Benoit Person For now, bin-wrappers overwrites GITPERLLIB. If we want to chain to those scripts and define GITPERLLIB before, our changes will be discarded. This patch makes the bin-wrappers prepend their modifications to GITPERLLIB rather than redefining it. It also unset GITPERLLIB in the test-suite to prevent broken $GITPERLLIB in the user's configuration from interfering with the testsuite. The codes using GIT_TEMPLATE_DIR and GIT_TEXTDOMAINDIR handle only one path in each of this variable so this new behavior would be useless on those variables. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- t/test-lib.sh | 1 + wrap-for-bin.sh | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index eff3a65..43e2a39 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -92,6 +92,7 @@ unset VISUAL EMAIL LANGUAGE COLUMNS $("$PERL_PATH" -e ' print join("\n", @vars); ') unset XDG_CONFIG_HOME +unset GITPERLLIB GIT_AUTHOR_EMAIL=aut...@example.com GIT_AUTHOR_NAME='A U Thor' GIT_COMMITTER_EMAIL=commit...@example.com diff --git a/wrap-for-bin.sh b/wrap-for-bin.sh index 53a8dd0..35b394a 100644 --- a/wrap-for-bin.sh +++ b/wrap-for-bin.sh @@ -14,7 +14,7 @@ else GIT_TEMPLATE_DIR='@@BUILD_DIR@@/templates/blt' export GIT_TEMPLATE_DIR fi -GITPERLLIB='@@BUILD_DIR@@/perl/blib/lib' +GITPERLLIB='@@BUILD_DIR@@/perl/blib/lib:'"${GITPERLLIB:+:$GITPERLLIB}" GIT_TEXTDOMAINDIR='@@BUILD_DIR@@/po/build/locale' PATH='@@BUILD_DIR@@/bin-wrappers:'"$PATH" export GIT_EXEC_PATH GITPERLLIB PATH GIT_TEXTDOMAINDIR -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 1/7] git-remote-mediawiki: Introduction of Git::Mediawiki.pm
From: Benoit Person Currently, the mw-to-git project contains only a remote helper (git-remote-mediawiki.perl). To improve the user experience while working with mediawiki remotes, new tools, designed for such cases, should be created. To achieve this goal, the project needs a way to share code between several scripts (remote helper, commands, ... ). A perl package offers the best way to handle such case: Each script can select what should be imported in its namespace. The package namespacing limits the use of side effects in the shared code. An alternate solution is to concatenate a "toolset" file with each *.perl when 'make'-ing the project. In that scheme, everything is imported in the script's namespace. Plus, files should be renamed in order to chain to Git's toplevel makefile. Hence, this solution is not acceptable. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 24 contrib/mw-to-git/Makefile | 24 +--- 2 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 000..805f42a --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,24 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..a6f8b24 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,18 +2,36 @@ # Copyright (C) 2013 # Matthieu Moy # -## Build git-remote-mediawiki +# To install, run Git's toplevel 'make install' then run: +# +# make install +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +install_pm: + install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + perlcritic: perlcritic -2 *.perl -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 0/7] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from v6: - Rewrite of bin-wrapper/git in 3/7 - Update wrap-for-bin.sh and make bin-wrappers chainable it removes the need for a GPLEXTEA since we only have to first define GITPERLLIB, the bin-wrappers will no longer overwrite it. (2/7) Maybe we should apply the same mechanism to all variables in wrap-for-bin.sh ? (there is no need for it right now though) - Replace the symlink created in the test suite (if git-remote-mediawiki was not installed) by the new bin-wrapper introduced in 3/4 (4/7). - Small typos in the commit messages changes from v5: - Move git bin-wrapper 'git' into bin-wrapper/ - Updates its GIT_ROOT_DIR accordingly - Updates the Makefile to use that new path in the comments changes from v4: - Rebase on latest master - Typos in commits messages and code - Comments in Makefile - Factoring the conversion from relatives links to absolute ones in `git mw preview` - Updating that "conversion" mechanism to not convert links with an anchor '#'. - git-mw should be executable now. changes from v3: - Rewrite all commit messages. - No more "\ No newline at end of file". - Rename GitMediawiki.pm into Git::Mediawiki.pm (so it moves GitMedawiki.pm into Git/Mediawiki.pm). - Remove from the Makefile the copy_pm target (see below 'Add a bin-wrapper'). - Use of 'install' insted of 'cp' in the Makefile. - Comment on the install_pm target in the Makefile. - Add a bin-wrapper for git to test scripts without 'make install'-ing them. - Move verbose option handling from previous v3-4/4 (introduction of preview tool) into v4-4/5 (introduction of git-mw). - Refactor some code into subroutines to clean the global 'preview' subroutine. - Rewrite some error messages to make them more concise while still giving the same amount of information. - Use 'remote.${remote_name}.mwIDcontent' instead of 'mediawiki.IDContent' as config item for the lookup ID used to combine template + new content. - Remove comments about what's going on in the preview subroutine. - Use 'clean_filename' (and not 'smudge_filename') in the preview tool to find the correct mediawiki page name based on a filename. - Remove space/tab mixup in the 'help' subroutine. changes from v2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from v1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (7): git-remote-mediawiki: Introduction of Git::Mediawiki.pm wrap-for-bin: Make bin-wrappers chainable git-remote-mediawiki: New git bin-wrapper for developement git-remote-mediawiki: Update tests to run with the new bin-wrapper git-remote-mediawiki: Factoring code between git-remote-mediawiki and Git::Mediawiki git-remote-mediawiki: Adding git-mw command git-remote-mediawiki: Add preview subcommand into git mw contrib/mw-to-git/Git/Mediawiki.pm | 100 contrib/mw-to-git/Makefile | 33 ++- contrib/mw-to-git/bin-wrapper/git | 14 ++ contrib/mw-to-git/git-mw.perl | 368 contrib/mw-to-git/git-remote-mediawik
[PATCH v7 2/7] wrap-for-bin: Make bin-wrappers chainable
From: Benoit Person For now, bin-wrappers (based on wrap-for-bin.sh) redefine some environnement variables (like $GITPERLLIB). If we want to chain to those scripts and define one of those variables before, our changes will be overwritten. This patch simply makes the bin-wrappers prepend their modifications rather than redefine the vars. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- wrap-for-bin.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wrap-for-bin.sh b/wrap-for-bin.sh index 53a8dd0..dbebe49 100644 --- a/wrap-for-bin.sh +++ b/wrap-for-bin.sh @@ -14,7 +14,7 @@ else GIT_TEMPLATE_DIR='@@BUILD_DIR@@/templates/blt' export GIT_TEMPLATE_DIR fi -GITPERLLIB='@@BUILD_DIR@@/perl/blib/lib' +GITPERLLIB='@@BUILD_DIR@@/perl/blib/lib:'"$GITPERLLIB" GIT_TEXTDOMAINDIR='@@BUILD_DIR@@/po/build/locale' PATH='@@BUILD_DIR@@/bin-wrappers:'"$PATH" export GIT_EXEC_PATH GITPERLLIB PATH GIT_TEXTDOMAINDIR -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 4/7] git-remote-mediawiki: Update tests to run with the new bin-wrapper
From: Benoit Person Until now, if git-remote-mediawiki was not installed, the test suite copied it to the toplevel directory. This solution pollutes the directory with untracked files. Plus, we would need to copy the new git-mw.perl file to test it too. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/t/test-gitmw-lib.sh | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh index bb76cee..ca6860f 100755 --- a/contrib/mw-to-git/t/test-gitmw-lib.sh +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh @@ -62,12 +62,8 @@ test_check_precond () { test_done fi - if [ ! -f "$GIT_BUILD_DIR"/git-remote-mediawiki ]; - then - echo "No remote mediawiki for git found. Copying it in git" - echo "cp $GIT_BUILD_DIR/contrib/mw-to-git/git-remote-mediawiki $GIT_BUILD_DIR/" - ln -s "$GIT_BUILD_DIR"/contrib/mw-to-git/git-remote-mediawiki "$GIT_BUILD_DIR" - fi + GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd "../.." && pwd) + PATH="$GIT_EXEC_PATH"'/bin-wrapper:'"$PATH" if [ ! -d "$WIKI_DIR_INST/$WIKI_DIR_NAME" ]; then -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 3/7] git-remote-mediawiki: New git bin-wrapper for developement
From: Benoit Person The introduction of the Git::Mediawiki package makes it impossible to test, without installation, git-remote-mediawiki and git-mw. Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the use of the developement version of the Git::Mediawiki package, bypassing its installed version if any. An alternate solution was to 'install' all the files required at each build but it pollutes the toplevel with untracked files. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 6 ++ contrib/mw-to-git/bin-wrapper/git | 14 ++ 2 files changed, 20 insertions(+) create mode 100755 contrib/mw-to-git/bin-wrapper/git diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index a6f8b24..c5e66df 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,6 +2,12 @@ # Copyright (C) 2013 # Matthieu Moy # +# To build and test: +# +# make: +# bin-wrapper/git mw preview Some_page.mw +# bin-wrapper/git clone mediawiki::http://example.com/wiki/ +# # To install, run Git's toplevel 'make install' then run: # # make install diff --git a/contrib/mw-to-git/bin-wrapper/git b/contrib/mw-to-git/bin-wrapper/git new file mode 100755 index 000..1aa51f7 --- /dev/null +++ b/contrib/mw-to-git/bin-wrapper/git @@ -0,0 +1,14 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +GIT_ROOT_DIR=../../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GITPERLLIB" +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$PATH" + +export GITPERLLIB PATH + +exec "${GIT_EXEC_PATH}/bin-wrappers/git" "$@" -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 6/7] git-remote-mediawiki: Adding git-mw command
From: Benoit Person For now, git-remote-mediawiki is only a remote-helper. This patch adds a new toolset script in which we will be able to build new tools for git-remote-mediawiki. This toolset uses a subcommand-mechanism to launch the proper action. For now only the 'help' subcommand is implemented. It also provides some generic code for the verbose and help command line options. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 ++--- contrib/mw-to-git/git-mw.perl | 60 +++ 2 files changed, 64 insertions(+), 3 deletions(-) create mode 100755 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index c5e66df..775cb07 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -14,6 +14,7 @@ GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -27,15 +28,15 @@ install_pm: install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100755 index 000..4a3e4a9 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} -- 1.8.3.1.590.gc07d91b -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v7 7/7] git-remote-mediawiki: Add preview subcommand into git mw
From: Benoit Person In the current state, a user of git-remote-mediawiki can edit the markup text locally, but has to push to the remote wiki to see how the page is rendererd. Add a new 'git mw preview' command that allows rendering the markup text on the remote wiki without actually pushing any change on the wiki. This uses Mediawiki's API to render the markup and inserts it in an actual HTML page from the wiki so that CSS can be rendered properly. Most links should work when the page exists on the remote. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 310 - 2 files changed, 311 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 47fe4f4..d13c4df 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 4a3e4a9..28df3ee 100755 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,6 +12,14 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); # By default, use UTF-8 to communicate with Git and the user binmode STDERR, ':encoding(UTF-8)'; @@ -26,9 +34,26 @@ sub v_print { return; } +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] ); # Search for sub-command @@ -47,6 +72,288 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's +upstream if --remote is not set. If that remote is not found or if it +is not a mediawiki, it lists all mediawiki remotes configured and asks +you to replay your command with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in +the current dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed +by the remote mediawiki and combined with a template retrieved from +the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it +in your default web browser if the option --autoload is present. + +OPTIONS: +-r , --remote +If the remote is a mediawiki, the template and the parse engine +used for the preview will be those of that remote. +If not, a list of valid remotes will be shown. + +-a, --autoload +Try to load the HTML output in a new tab (or new window) of your +default web browser. + +-o , --output +Change the HTML output filename. Default filename is based on the +input filename with its extension replaced by '.html'. + +-v, --verbose +Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($new_content, $template); + my $file_content; + + i
[PATCH v7 5/7] git-remote-mediawiki: Factoring code between git-remote-mediawiki and Git::Mediawiki
From: Benoit Person For now, Git::Mediawiki contains nothing. This first patch moves some of git-remote-mediawiki.perl's factorisable code into Git::Mediawiki. In the same time, it removes the side effects of that code and renames the fucntions and constants moved to expose a better API. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 805f42a..47fe4f4 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 71baf8a..e40c034 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use consta
[PATCH v6 0/5] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from v5: - Move git bin-wrapper 'git' into bin-wrapper/ - Updates its GIT_ROOT_DIR accordingly - Updates the Makefile to use that new path in the comments changes from v4: - Rebase on latest master - Typos in commits messages and code - Comments in Makefile - Factoring the conversion from relatives links to absolute ones in `git mw preview` - Updating that "conversion" mechanism to not convert links with an anchor '#'. - git-mw should be executable now. changes from v3: - Rewrite all commit messages. - No more "\ No newline at end of file". - Rename GitMediawiki.pm into Git::Mediawiki.pm (so it moves GitMedawiki.pm into Git/Mediawiki.pm). - Remove from the Makefile the copy_pm target (see below 'Add a bin-wrapper'). - Use of 'install' insted of 'cp' in the Makefile. - Comment on the install_pm target in the Makefile. - Add a bin-wrapper for git to test scripts without 'make install'-ing them. - Move verbose option handling from previous v3-4/4 (introduction of preview tool) into v4-4/5 (introduction of git-mw). - Refactor some code into subroutines to clean the global 'preview' subroutine. - Rewrite some error messages to make them more concise while still giving the same amount of information. - Use 'remote.${remote_name}.mwIDcontent' instead of 'mediawiki.IDContent' as config item for the lookup ID used to combine template + new content. - Remove comments about what's going on in the preview subroutine. - Use 'clean_filename' (and not 'smudge_filename') in the preview tool to find the correct mediawiki page name based on a filename. - Remove space/tab mixup in the 'help' subroutine. changes from v2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from v1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name This serie is based on the 'master' branch merged with célestin's patch series. [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (5): git-remote-mediawiki: Introduction of Git::Mediawiki.pm git-remote-mediawiki: new git bin-wrapper for developement git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki git-remote-mediawiki: Adding git-mw command git-remote-mediawiki: Add preview subcommand into git mw. contrib/mw-to-git/Git/Mediawiki.pm | 100 contrib/mw-to-git/Makefile | 33 ++- contrib/mw-to-git/bin-wrapper/git | 27 ++ contrib/mw-to-git/git-mw.perl | 368 contrib/mw-to-git/git-remote-mediawiki.perl | 85 +-- 5 files changed, 537 insertions(+), 76 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm create mode 100755 contrib/mw-to-git/bin-wrapper/git create mode 100755 contrib/mw-to-git/git-mw.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v6 5/5] git-remote-mediawiki: Add preview subcommand into git mw.
From: Benoit Person In the current state, a user of git-remote-mediawiki can edit the markup text locally, but has to push to the remote wiki to see how the page is rendererd. Add a new 'git mw preview' command that allows rendering the markup text on the remote wiki without actually pushing any change on the wiki. This uses Mediawiki's API to render the markup and inserts it in an actual HTML page from the wiki so that CSS can be rendered properly. Most links should work when the page exists on the remote. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 310 - 2 files changed, 311 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 47fe4f4..d13c4df 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 4a3e4a9..28df3ee 100755 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,6 +12,14 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); # By default, use UTF-8 to communicate with Git and the user binmode STDERR, ':encoding(UTF-8)'; @@ -26,9 +34,26 @@ sub v_print { return; } +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] ); # Search for sub-command @@ -47,6 +72,288 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's +upstream if --remote is not set. If that remote is not found or if it +is not a mediawiki, it lists all mediawiki remotes configured and asks +you to replay your command with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in +the current dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed +by the remote mediawiki and combined with a template retrieved from +the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it +in your default web browser if the option --autoload is present. + +OPTIONS: +-r , --remote +If the remote is a mediawiki, the template and the parse engine +used for the preview will be those of that remote. +If not, a list of valid remotes will be shown. + +-a, --autoload +Try to load the HTML output in a new tab (or new window) of your +default web browser. + +-o , --output +Change the HTML output filename. Default filename is based on the +input filename with its extension replaced by '.html'. + +-v, --verbose +Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($new_content, $template); + my $file_content; + + i
[PATCH v6 2/5] git-remote-mediawiki: new git bin-wrapper for developement
From: Benoit Person The introduction of the Git::Mediawiki package makes it impossible to test, without installation, git-remote-mediawiki and git-mw. Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the use of the developement version of the Git::Mediawiki package, bypassing its installed version if any. An alternate solution was to 'install' all the files required at each build but it pollutes the toplevel with untracked files. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 6 ++ contrib/mw-to-git/bin-wrapper/git | 27 +++ 2 files changed, 33 insertions(+) create mode 100755 contrib/mw-to-git/bin-wrapper/git diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index a6f8b24..c5e66df 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,6 +2,12 @@ # Copyright (C) 2013 # Matthieu Moy # +# To build and test: +# +# make: +# bin-wrapper/git mw preview Some_page.mw +# bin-wrapper/git clone mediawiki::http://example.com/wiki/ +# # To install, run Git's toplevel 'make install' then run: # # make install diff --git a/contrib/mw-to-git/bin-wrapper/git b/contrib/mw-to-git/bin-wrapper/git new file mode 100755 index 000..aa714a5 --- /dev/null +++ b/contrib/mw-to-git/bin-wrapper/git @@ -0,0 +1,27 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +# based on $GIT_ROOT_DIR/wrap-for-bin.sh + + +GIT_ROOT_DIR=../../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +echo $GIT_EXEC_PATH + +if test -n "$NO_SET_GIT_TEMPLATE_DIR" +then + unset GIT_TEMPLATE_DIR +else + GIT_TEMPLATE_DIR="$GIT_EXEC_PATH"'/templates/blt' + export GIT_TEMPLATE_DIR +fi +# Hack to make the `use lib` call works with multiple paths +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/perl/blib/lib' +GIT_TEXTDOMAINDIR="$GIT_EXEC_PATH"'/po/build/locale' +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/bin-wrappers:'"$PATH" +export GIT_EXEC_PATH GITPERLLIB PATH GIT_TEXTDOMAINDIR + +exec "${GIT_EXEC_PATH}/git" "$@" -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v6 4/5] git-remote-mediawiki: Adding git-mw command
From: Benoit Person For now, git-remote-mediawiki is only a remote-helper. This patch adds a new toolset script in which we will be able to build new tools for git-remote-mediawiki. This toolset uses a subcommand-mechanism to launch the proper action. For now only the 'help' subcommand is implemented. It also provides some generic code for the verbose and help command line options. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 ++--- contrib/mw-to-git/git-mw.perl | 60 +++ 2 files changed, 64 insertions(+), 3 deletions(-) create mode 100755 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index c5e66df..775cb07 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -14,6 +14,7 @@ GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -27,15 +28,15 @@ install_pm: install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100755 index 000..4a3e4a9 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v6 1/5] git-remote-mediawiki: Introduction of Git::Mediawiki.pm
From: Benoit Person Currently, the mw-to-git project contains only a remote helper (git-remote-mediawiki.perl). To improve the user experience while working with mediawiki remotes, new tools, designed for such cases, should be created. To achieve this goal, the project needs a way to share code between several scripts (remote helper, commands, ... ). A perl package offers the best way to handle such case: Each script can select what should be imported in its namespace. The package namespacing limits the use of side effects in the shared code. An alternate solution is to concatenate a "toolset" file with each *.perl when 'make'-ing the project. In that scheme, everything is imported in the script's namespace. Plus, files should be renamed in order to chain to Git's toplevel makefile. Hence, this solution is not acceptable. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 24 contrib/mw-to-git/Makefile | 24 +--- 2 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 000..805f42a --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,24 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..a6f8b24 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,18 +2,36 @@ # Copyright (C) 2013 # Matthieu Moy # -## Build git-remote-mediawiki +# To install, run Git's toplevel 'make install' then run: +# +# make install +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +install_pm: + install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + perlcritic: perlcritic -2 *.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v6 3/5] git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki
From: Benoit Person For now, Git::Mediawiki contains nothing. This first patch moves some of git-remote-mediawiki.perl's factorisable code into Git::Mediawiki. In the same time, it removes the side effects of that code and renames the fucntions and constants moved to expose a better API. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 805f42a..47fe4f4 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 71baf8a..e40c034 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use consta
[PATCH v5 1/5] git-remote-mediawiki: Introduction of Git::Mediawiki.pm
From: Benoit Person Currently, the mw-to-git project contains only a remote helper (git-remote-mediawiki.perl). To improve the user experience while working with mediawiki remotes, new tools, designed for such cases, should be created. To achieve this goal, the project needs a way to share code between several scripts (remote helper, commands, ... ). A perl package offers the best way to handle such case: Each script can select what should be imported in its namespace. The package namespacing limits the use of side effects in the shared code. An alternate solution is to concatenate a "toolset" file with each *.perl when 'make'-ing the project. In that scheme, everything is imported in the script's namespace. Plus, files should be renamed in order to chain to Git's toplevel makefile. Hence, this solution is not acceptable. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 24 contrib/mw-to-git/Makefile | 24 +--- 2 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 000..805f42a --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,24 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..a6f8b24 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,18 +2,36 @@ # Copyright (C) 2013 # Matthieu Moy # -## Build git-remote-mediawiki +# To install, run Git's toplevel 'make install' then run: +# +# make install +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +install_pm: + install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + perlcritic: perlcritic -2 *.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v5 5/5] git-remote-mediawiki: Add preview subcommand into git mw.
From: Benoit Person In the current state, a user of git-remote-mediawiki can edit the markup text locally, but has to push to the remote wiki to see how the page is rendererd. Add a new 'git mw preview' command that allows rendering the markup text on the remote wiki without actually pushing any change on the wiki. This uses Mediawiki's API to render the markup and inserts it in an actual HTML page from the wiki so that CSS can be rendered properly. Most links should work when the page exists on the remote. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 310 - 2 files changed, 311 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 47fe4f4..d13c4df 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 4a3e4a9..28df3ee 100755 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,6 +12,14 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); # By default, use UTF-8 to communicate with Git and the user binmode STDERR, ':encoding(UTF-8)'; @@ -26,9 +34,26 @@ sub v_print { return; } +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] ); # Search for sub-command @@ -47,6 +72,288 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's +upstream if --remote is not set. If that remote is not found or if it +is not a mediawiki, it lists all mediawiki remotes configured and asks +you to replay your command with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in +the current dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed +by the remote mediawiki and combined with a template retrieved from +the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it +in your default web browser if the option --autoload is present. + +OPTIONS: +-r , --remote +If the remote is a mediawiki, the template and the parse engine +used for the preview will be those of that remote. +If not, a list of valid remotes will be shown. + +-a, --autoload +Try to load the HTML output in a new tab (or new window) of your +default web browser. + +-o , --output +Change the HTML output filename. Default filename is based on the +input filename with its extension replaced by '.html'. + +-v, --verbose +Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($new_content, $template); + my $file_content; + + i
[PATCH v5 3/5] git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki
From: Benoit Person For now, Git::Mediawiki contains nothing. This first patch moves some of git-remote-mediawiki.perl's factorisable code into Git::Mediawiki. In the same time, it removes the side effects of that code and renames the fucntions and constants moved to expose a better API. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 805f42a..47fe4f4 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 71baf8a..e40c034 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use consta
[PATCH v5 4/5] git-remote-mediawiki: Adding git-mw command
From: Benoit Person For now, git-remote-mediawiki is only a remote-helper. This patch adds a new toolset script in which we will be able to build new tools for git-remote-mediawiki. This toolset uses a subcommand-mechanism to launch the proper action. For now only the 'help' subcommand is implemented. It also provides some generic code for the verbose and help command line options. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 ++--- contrib/mw-to-git/git-mw.perl | 60 +++ 2 files changed, 64 insertions(+), 3 deletions(-) create mode 100755 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 2a80d56..37afbbf 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -14,6 +14,7 @@ GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -27,15 +28,15 @@ install_pm: install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100755 index 000..4a3e4a9 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v5 0/5] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from v4: - Rebase on latest master - Typos in commits messages and code - Comments in Makefile - Factoring the conversion from relatives links to absolute ones in `git mw preview` - Updating that "conversion" mechanism to not convert links with an anchor '#'. - git-mw should be executable now. changes from v3: - Rewrite all commit messages. - No more "\ No newline at end of file". - Rename GitMediawiki.pm into Git::Mediawiki.pm (so it moves GitMedawiki.pm into Git/Mediawiki.pm). - Remove from the Makefile the copy_pm target (see below 'Add a bin-wrapper'). - Use of 'install' insted of 'cp' in the Makefile. - Comment on the install_pm target in the Makefile. - Add a bin-wrapper for git to test scripts without 'make install'-ing them. - Move verbose option handling from previous v3-4/4 (introduction of preview tool) into v4-4/5 (introduction of git-mw). - Refactor some code into subroutines to clean the global 'preview' subroutine. - Rewrite some error messages to make them more concise while still giving the same amount of information. - Use 'remote.${remote_name}.mwIDcontent' instead of 'mediawiki.IDContent' as config item for the lookup ID used to combine template + new content. - Remove comments about what's going on in the preview subroutine. - Use 'clean_filename' (and not 'smudge_filename') in the preview tool to find the correct mediawiki page name based on a filename. - Remove space/tab mixup in the 'help' subroutine. changes from v2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from v1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name This serie is based on the 'master' branch merged with célestin's patch series. [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (5): git-remote-mediawiki: Introduction of Git::Mediawiki.pm git-remote-mediawiki: new git bin-wrapper for developement git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki git-remote-mediawiki: Adding git-mw command git-remote-mediawiki: Add preview subcommand into git mw. contrib/mw-to-git/Git/Mediawiki.pm | 100 contrib/mw-to-git/Makefile | 33 ++- contrib/mw-to-git/git | 25 ++ contrib/mw-to-git/git-mw.perl | 368 contrib/mw-to-git/git-remote-mediawiki.perl | 85 +-- 5 files changed, 535 insertions(+), 76 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm create mode 100755 contrib/mw-to-git/git create mode 100755 contrib/mw-to-git/git-mw.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v5 2/5] git-remote-mediawiki: new git bin-wrapper for developement
From: Benoit Person The introduction of the Git::Mediawiki package makes it impossible to test, without installation, git-remote-mediawiki and git-mw. Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the use of the developement version of the Git::Mediawiki package, bypassing its installed version if any. An alternate solution was to 'install' all the files required at each build but it pollutes the toplevel with untracked files. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile | 6 ++ contrib/mw-to-git/git | 25 + 2 files changed, 31 insertions(+) create mode 100755 contrib/mw-to-git/git diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index a6f8b24..2a80d56 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -2,6 +2,12 @@ # Copyright (C) 2013 # Matthieu Moy # +# To build and test: +# +# make: +# ./git mw preview Some_page.mw +# ./git clone mediawiki::http://example.com/wiki/ +# # To install, run Git's toplevel 'make install' then run: # # make install diff --git a/contrib/mw-to-git/git b/contrib/mw-to-git/git new file mode 100755 index 000..25b60ad --- /dev/null +++ b/contrib/mw-to-git/git @@ -0,0 +1,25 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +# based on $GIT_ROOT_DIR/wrap-for-bin.sh + + +GIT_ROOT_DIR=../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +if test -n "$NO_SET_GIT_TEMPLATE_DIR" +then + unset GIT_TEMPLATE_DIR +else + GIT_TEMPLATE_DIR="$GIT_EXEC_PATH"'/templates/blt' + export GIT_TEMPLATE_DIR +fi +# Hack to make the `use lib` call works with multiple paths +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/perl/blib/lib' +GIT_TEXTDOMAINDIR="$GIT_EXEC_PATH"'/po/build/locale' +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/bin-wrappers:'"$PATH" +export GIT_EXEC_PATH GITPERLLIB PATH GIT_TEXTDOMAINDIR + +exec "${GIT_EXEC_PATH}/git" "$@" \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC 3/4] git-mw: Adding git-mw.perl script
Le 2013-06-24 18:56, Matthieu Moy a écrit : Junio C Hamano writes: Matthieu Moy writes: benoit.per...@ensimag.fr writes: diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100644 index 000..a2f0aa1 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl *.perl scripts are usually executable in Git's tree (although it's usually better to run the non-*.perl version). Good eyes. But if we encourage people to run non-*.perl version, perhaps we should drop the executable bit from the source, no? But by default, I'd say consistency is most important so if other *.perl are executable, we should do the same (otherwise my "ls" shows different colors and it's ugly ;-) ). So it's really a *nice* catch then :) . But it may make sense to change the convention, i.e. run a "chmod -x *.perl" in Git's tree (in any case, people can still run "perl foo.perl"). For what I've seen so far of git.git, the best way would be to make it executable in this patch serie and send another patch applying that 'chmod -x'-thingy ? -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 3/5] git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki
From: Benoit Person For now, Git::Mediawiki contains nothing. This first patch moves some of git-remote-mediawiki.perl's factorisable code into Git::Mediawiki. In the same time, it removes the side effects of that code and renames the fucntions and constants moved to expose a better API. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 805f42a..47fe4f4 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 9ff45fd..a336887 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use Git::Mediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)'; use URI::Escape; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => '%2F'; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are replaced with this content. use consta
[PATCH v4 4/5] git-remote-mediawiki: Adding git-mw command
From: Benoit Person For now, git-remote-mediawiki is only a remote-helper. This patch adds a new toolset script in which we will be able to build new tools for git-remote-mediawiki. This toolset uses a subcommand-mechanism to launch the proper action. For now only the 'help' subcommand is implemented. It also provides some generic code for the verbose and help command line options. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 ++--- contrib/mw-to-git/git-mw.perl | 60 +++ 2 files changed, 64 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index becd322..e2ae798 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -6,6 +6,7 @@ GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -20,15 +21,15 @@ install_pm: install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100644 index 000..4a3e4a9 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +# Global parameters +my $verbose = 0; +sub v_print { + if ($verbose) { + return print {*STDERR} @_; + } + return; +} + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV-1) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}, + 'verbose|v' => \$verbose); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 2/5] git-remote-mediawiki: new git bin-wrapper for developement
From: Benoit Person The introduction of the Git::Mediawiki package makes it impossible to test, without installation, git-remote-mediawiki and git-mw. Using a git bin-wrapper enables us to define proper $GITPERLLIB to force the use of the developement version of the Git::Mediawiki package, bypassing its installed version if any. An alternate solution was to 'install' all the files required at each build but it pollutes the toplevel with untracked files. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/git | 25 + 1 file changed, 25 insertions(+) create mode 100755 contrib/mw-to-git/git diff --git a/contrib/mw-to-git/git b/contrib/mw-to-git/git new file mode 100755 index 000..25b60ad --- /dev/null +++ b/contrib/mw-to-git/git @@ -0,0 +1,25 @@ +#!/bin/sh + +# git executable wrapper script for Git-Mediawiki to run tests without +# installing all the scripts and perl packages. + +# based on $GIT_ROOT_DIR/wrap-for-bin.sh + + +GIT_ROOT_DIR=../.. +GIT_EXEC_PATH=$(cd "$(dirname "$0")" && cd ${GIT_ROOT_DIR} && pwd) + +if test -n "$NO_SET_GIT_TEMPLATE_DIR" +then + unset GIT_TEMPLATE_DIR +else + GIT_TEMPLATE_DIR="$GIT_EXEC_PATH"'/templates/blt' + export GIT_TEMPLATE_DIR +fi +# Hack to make the `use lib` call works with multiple paths +GITPERLLIB="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/perl/blib/lib' +GIT_TEXTDOMAINDIR="$GIT_EXEC_PATH"'/po/build/locale' +PATH="$GIT_EXEC_PATH"'/contrib/mw-to-git:'"$GIT_EXEC_PATH"'/bin-wrappers:'"$PATH" +export GIT_EXEC_PATH GITPERLLIB PATH GIT_TEXTDOMAINDIR + +exec "${GIT_EXEC_PATH}/git" "$@" \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 1/5] git-remote-mediawiki: Introduction of Git::Mediawiki.pm
From: Benoit Person Currently, the mw-to-git project contains only a remote helper (git-remote-mediawiki.perl). To inmprove the user experience while working with mediawiki remotes, new tools, designed for such cases, should be created. To achieve this goal, the project needs a way to share code between several scripts (remote helper, commands, ... ). A perl package offer the best way to handle such case: Each script can select what should be imported in its namespace. The package namespacing limits the use of side effects in the shared code. An alternate solution is to concatenate a "toolset" file with each *.perl when 'make'-ing the project. In that scheme, everything is imported in the script's namespace. Plus, files should be renamed in order to chain to Git's toplevel makefile. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 24 contrib/mw-to-git/Makefile | 21 +++-- 2 files changed, 43 insertions(+), 2 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm new file mode 100644 index 000..805f42a --- /dev/null +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -0,0 +1,24 @@ +package Git::Mediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..becd322 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -4,16 +4,33 @@ # ## Build git-remote-mediawiki +GIT_MEDIAWIKI_PM=Git/Mediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +# Run 'make install' from Git's toplevel before using this +install_pm: + install $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) + perlcritic: perlcritic -2 *.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v4 5/5] git-remote-mediawiki: Add preview subcommand into git mw.
From: Benoit Person In the current state, a user of git-remote-mediawiki can edit the markup text locally, but has to push to the remote wiki to see how the page is rendererd. Add a new 'git mw preview' command that allows rendering the markup text on the remote wiki without actually pushing any change on the wiki. This uses Mediawiki's API to render the markup and inserts it in an actual HTML page from the wiki so that CSS can be rendered properly. Most links should work when the page exists on the remote. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Git/Mediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 301 - 2 files changed, 302 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/Git/Mediawiki.pm b/contrib/mw-to-git/Git/Mediawiki.pm index 47fe4f4..d13c4df 100644 --- a/contrib/mw-to-git/Git/Mediawiki.pm +++ b/contrib/mw-to-git/Git/Mediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 4a3e4a9..d4f412a 100644 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,6 +12,14 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use Git::Mediawiki qw(clean_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); # By default, use UTF-8 to communicate with Git and the user binmode STDERR, ':encoding(UTF-8)'; @@ -26,9 +34,26 @@ sub v_print { return; } +# Preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +sub file { + $file_name = shift; + return $file_name; +} + my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload + }, \&preview_help] ); # Search for sub-command @@ -47,6 +72,279 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +# @TODO : add documentation for verbose option +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's upstream +if --remote is not set. If that remote is not found or if it is not a mediawiki, +it lists all mediawiki remotes configured and asks you to replay your command +with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in the current +dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed by the +distant mediawiki and combined with a template retrieved from the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it in your +default web browser if the option --autoload is present. + +OPTIONS: + -r , --remote + If the remote is a mediawiki, the template and the parse engine used for + the preview will be those of that remote. + If not, a list of valid remotes will be shown. + + -a, --autoload + Try to load the HTML output in a new tab (or new window) of your default + web browser. + + -o , --output + Change the HTML output filename. Default filename is based on the input + filename with its extension replaced by '.html'. + + -v, --verbose + Show more information on what's going on under the hood. +END + exit; +} + +sub preview { + my $wi
[PATCH v4 0/5] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from v3: - Rewrite all commit messages. - No more "\ No newline at end of file". - Rename GitMediawiki.pm into Git::Mediawiki.pm (so it moves GitMedawiki.pm into Git/Mediawiki.pm). - Remove from the Makefile the copy_pm target (see below 'Add a bin-wrapper'). - Use of 'install' insted of 'cp' in the Makefile. - Comment on the install_pm target in the Makefile. - Add a bin-wrapper for git to test scripts without 'make install'-ing them. - Move verbose option handling from previous v3-4/4 (introduction of preview tool) into v4-4/5 (introduction of git-mw). - Refactor some code into subroutines to clean the global 'preview' subroutine. - Rewrite some error messages to make them more concise while still giving the same amount of information. - Use 'remote.${remote_name}.mwIDcontent' instead of 'mediawiki.IDContent' as config item for the lookup ID used to combine template + new content. - Remove comments about what's going on in the preview subroutine. - Use 'clean_filename' (and not 'smudge_filename') in the preview tool to find the correct mediawiki page name based on a filename. - Remove space/tab mixup in the 'help' subroutine. changes from v2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from v1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name This serie is based on the 'master' branch merged with célestin's patch series. [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (5): git-remote-mediawiki: Introduction of Git::Mediawiki.pm git-remote-mediawiki: new git bin-wrapper for developement git-remote-mediawiki: factoring code between git-remote-mediawiki and Git::Mediawiki git-remote-mediawiki: Adding git-mw command git-remote-mediawiki: Add preview subcommand into git mw. contrib/mw-to-git/Git/Mediawiki.pm | 100 contrib/mw-to-git/Makefile | 24 +- contrib/mw-to-git/git | 25 ++ contrib/mw-to-git/git-mw.perl | 359 contrib/mw-to-git/git-remote-mediawiki.perl | 85 +-- 5 files changed, 518 insertions(+), 75 deletions(-) create mode 100644 contrib/mw-to-git/Git/Mediawiki.pm create mode 100755 contrib/mw-to-git/git create mode 100644 contrib/mw-to-git/git-mw.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V3 3/4] git-mw: Adding git-mw command
From: Benoit Person Add the new git-mw command, with its 'help' subcommand as an example. Argument parsing and subcommand choice is based on git-svn.perl. Update Makefile to build, install and clean both scripts now (git-mw.perl and git-remote-mediawiki.perl). Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- changes from V2: - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. contrib/mw-to-git/Makefile| 7 --- contrib/mw-to-git/git-mw.perl | 46 +++ 2 files changed, 50 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index b0c7cf2..fe02c7e 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -6,6 +6,7 @@ GIT_MEDIAWIKI_PM=GitMediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -22,15 +23,15 @@ install_pm: cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR) build: copy_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \ $(GIT_ROOT_DIR)/perl/blib/lib/$(GIT_MEDIAWIKI_PM) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100644 index 000..320c00e --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,46 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (0..@ARGV) { + if (defined $commands{$ARGV[$_]}) { + $cmd = $commands{$ARGV[$_]}; + splice @ARGV, $_, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print {*STDOUT} <<'END'; +usage: git mw + +git mw commands are: +helpDisplay help information about git mw +END + exit; +} \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V3 2/4] git-mw: Move some functions from git-remote-mediawiki.perl to GitMediawiki.pm
From: Benoit Person Move and rename subs 'mediawiki_clean_filename' into 'clean_filename', 'mediawiki_smudge_filename' into 'smudge_filename' and 'mw_connect_maybe' into 'connect_maybe' since we have a cleaner namespace in a perl module. Move constants 'EMPTY', 'SLASH_REPLACEMENT' and 'HTTP_CODE_OK'. Remove side effects in those functions to provide a cleaner API. Update git-remote-mediawiki for those changes. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- changes from the V2: - Move more constants to GitMediawiki.pm. - Remove the encapsulation of Git::config calls into a git_cmd_try one. contrib/mw-to-git/GitMediawiki.pm | 77 +- contrib/mw-to-git/git-remote-mediawiki.perl | 85 + 2 files changed, 89 insertions(+), 73 deletions(-) diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm index 8a0ffc7..beae6d0 100644 --- a/contrib/mw-to-git/GitMediawiki.pm +++ b/contrib/mw-to-git/GitMediawiki.pm @@ -18,7 +18,82 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +# Used to test for empty strings +use constant EMPTY => q{}; + +# HTTP codes +use constant HTTP_CODE_OK => 200; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain"); + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words \ No newline at end of file diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 9ff45fd..3b6422b 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -14,6 +14,8 @@ use strict; use MediaWiki::API; use Git; +use GitMediawiki qw(clean_filename smudge_filename connect_maybe + EMPTY HTTP_CODE_OK); use DateTime::Format::ISO8601; use warnings; @@ -23,9 +25,6 @@ binmode STDOUT, &
[PATCH V3 4/4] git-mw: Add preview subcommand into git mw.
From: Benoit Person Add the subcommand to 'git-mw.perl'. Add a new constant in GitMediawiki.pm 'HTTP_CODE_PAGE_NOT_FOUND'. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- changes from V2: - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . contrib/mw-to-git/GitMediawiki.pm | 3 +- contrib/mw-to-git/git-mw.perl | 303 +- 2 files changed, 304 insertions(+), 2 deletions(-) diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm index beae6d0..d1f2c41 100644 --- a/contrib/mw-to-git/GitMediawiki.pm +++ b/contrib/mw-to-git/GitMediawiki.pm @@ -19,7 +19,7 @@ require Exporter; # Methods which can be called as standalone functions as well: @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe - EMPTY HTTP_CODE_OK); + EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND); } # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced @@ -30,6 +30,7 @@ use constant EMPTY => q{}; # HTTP codes use constant HTTP_CODE_OK => 200; +use constant HTTP_CODE_PAGE_NOT_FOUND => 404; sub clean_filename { my $filename = shift; diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index 320c00e..0b83108 100644 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,10 +12,41 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use LWP::UserAgent; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use GitMediawiki qw(smudge_filename connect_maybe + EMPTY HTTP_CODE_PAGE_NOT_FOUND); + +# By default, use UTF-8 to communicate with Git and the user +binmode STDERR, ':encoding(UTF-8)'; +binmode STDOUT, ':encoding(UTF-8)'; + +#preview parameters +my $file_name = EMPTY; +my $remote_name = EMPTY; +my $preview_file_name = EMPTY; +my $autoload = 0; +my $verbose = 0; +sub file { + $file_name = shift; + return $file_name; +} my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload, + 'verbose|v' => \$verbose + }, \&preview_help] ); # Search for sub-command @@ -33,6 +64,275 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +# @TODO : add documentation for verbose option +sub preview_help { + print {*STDOUT} <<'END'; +USAGE: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] [--verbose|-v] + | + +DESCRIPTION: +Preview is an utiliy to preview local content of a mediawiki repo as if it was +pushed on the remote. + +For that, preview searches for the remote name of the current branch's upstream +if --remote is not set. If that remote is not found or if it is not a mediawiki, +it lists all mediawiki remotes configured and asks you to replay your command +with the --remote option set properly. + +Then, it searches for a file named 'filename'. If it's not found in the current +dir, it will assume it's a blob. + +The content retrieved in the file (or in the blob) will then be parsed by the +distant mediawiki and combined with a template retrieved from the mediawiki. + +Finally, preview will save the HTML result in a file. and autoload it in your +default web browser if the option --autoload is present. + +OPTIONS: + -r , --remote + If the remote is a mediawiki, the template and the parse engine used for + the preview will be those of that remote. + If not, a list of valid remotes will be shown. + + -a, --autoloa
[PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm
From: Benoit Person GitMediawiki.pm enable code factoring between several scripts in mw-to-git. To make it usable in scripts (ie: accessible in @INC) it adds two targets (copy_pm & install_pm) into the Makefile, one for tests and one for installation. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- changes from the V2: - Add a way to test, without installation, code that uses GitMediawiki.pm. contrib/mw-to-git/GitMediawiki.pm | 24 contrib/mw-to-git/Makefile| 26 +++--- 2 files changed, 47 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/GitMediawiki.pm diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm new file mode 100644 index 000..8a0ffc7 --- /dev/null +++ b/contrib/mw-to-git/GitMediawiki.pm @@ -0,0 +1,24 @@ +package GitMediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words \ No newline at end of file diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index 1fb2424..b0c7cf2 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -4,16 +4,36 @@ # ## Build git-remote-mediawiki +GIT_MEDIAWIKI_PM=GitMediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +copy_pm: + cp $(GIT_MEDIAWIKI_PM) $(GIT_ROOT_DIR)/perl/blib/lib/ + +install_pm: + cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR) + +build: copy_pm + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +install-perl-script + +clean: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \ + $(GIT_ROOT_DIR)/perl/blib/lib/$(GIT_MEDIAWIKI_PM) + perlcritic: - perlcritic -2 *.perl + perlcritic -2 *.perl \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V3 0/4] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. changes from V2: - Add a way to test, without installation, code that uses GitMediawiki.pm. - Move more constants to GitMediawiki.pm - Remove the encapsulation of Git::config calls into a git_cmd_try one. - Remove the --blob option, distinction between files and blobs is now automatic. - Add a --verbose option to output more information on what's going on. - Rewrote the doc and the commit message. - Rewrote of the template retrieving code (see 'get_template' sub). - Use a configuration variable to define the content ID search in the template. Default value set as 'bodyContent' since it seems more standard than 'mw-content-text'. - Final content is now saved as utf-8 to solve encoding issues. - Perlcritic changes: - Update for loops style to a more perlish one. - All 'print's specify their output streams. --> Same useless warnings left in git-remote-mediawiki.perl after célestin's work and git-mw.perl after this patch :) . changes from V1: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name This serie is based on the 'master' branch merged with célestin's patch series. [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (4): git-mw: Introduction of GitMediawiki.pm git-mw: Move some functions from git-remote-mediawiki.perl to GitMediawiki.pm git-mw: Adding git-mw command git-mw: Add preview subcommand into git mw. contrib/mw-to-git/GitMediawiki.pm | 100 contrib/mw-to-git/Makefile | 29 ++- contrib/mw-to-git/git-mw.perl | 347 contrib/mw-to-git/git-remote-mediawiki.perl | 85 ++- 4 files changed, 485 insertions(+), 76 deletions(-) create mode 100644 contrib/mw-to-git/GitMediawiki.pm create mode 100644 contrib/mw-to-git/git-mw.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH V2] git-remote-mediawiki: remove hardcoded version number in the test suite
From: Benoit Person Updates the code to make it more easy to switch mediawiki version when testing. Before that, the version number was partly hardcoded, partly in a var. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/t/test-gitmw-lib.sh | 19 ++- contrib/mw-to-git/t/test.config | 4 +++- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh index 3b2cfac..bb76cee 100755 --- a/contrib/mw-to-git/t/test-gitmw-lib.sh +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh @@ -336,20 +336,21 @@ wiki_install () { fi # Fetch MediaWiki's archive if not already present in the TMP directory + MW_FILENAME="mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz" cd "$TMP" - if [ ! -f "$MW_VERSION.tar.gz" ] ; then - echo "Downloading $MW_VERSION sources ..." - wget "http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0.tar.gz"; || + if [ ! -f $MW_FILENAME ] ; then + echo "Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ..." + wget "http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/$MW_FILENAME"; || error "Unable to download "\ - "http://download.wikimedia.org/mediawiki/1.19/"\ - "mediawiki-1.19.0.tar.gz. "\ + "http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/"\ + "$MW_FILENAME. "\ "Please fix your connection and launch the script again." - echo "$MW_VERSION.tar.gz downloaded in `pwd`. "\ + echo "$MW_FILENAME downloaded in `pwd`. "\ "You can delete it later if you want." else - echo "Reusing existing $MW_VERSION.tar.gz downloaded in `pwd`." + echo "Reusing existing $MW_FILENAME downloaded in `pwd`." fi - archive_abs_path=$(pwd)/"$MW_VERSION.tar.gz" + archive_abs_path=$(pwd)/$MW_FILENAME cd "$WIKI_DIR_INST/$WIKI_DIR_NAME/" || error "can't cd to $WIKI_DIR_INST/$WIKI_DIR_NAME/" tar xzf "$archive_abs_path" --strip-components=1 || @@ -431,5 +432,5 @@ wiki_delete () { # Delete the wiki's SQLite database rm -f "$TMP/$DB_FILE" || error "Database $TMP/$DB_FILE could not be deleted." rm -f "$FILES_FOLDER/$DB_FILE" - rm -rf "$TMP/$MW_VERSION" + rm -rf "$TMP/mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz" } diff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config index 958b37b..4cfebe9 100644 --- a/contrib/mw-to-git/t/test.config +++ b/contrib/mw-to-git/t/test.config @@ -30,6 +30,8 @@ WEB_WWW=$WEB/www # The variables below are used by the script to install a wiki. # You should not modify these unless you are modifying the script itself. -MW_VERSION=mediawiki-1.19.0 +# tested versions: 1.19.X -> 1.21.1 +MW_VERSION_MAJOR=1.21 +MW_VERSION_MINOR=1 FILES_FOLDER=install-wiki DB_INSTALL_SCRIPT=db_install.php -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-remote-mediawiki: remove hardcoded version number in the test suite
From: Benoit Person Updates the code to make it more easy to switch mediawiki version when testing. Before that, the version number was partly hardcoded, partly in a var. Maybe I should add a warning that the installation procedure may not work in the future ? It seems to work for the range 1.19.X - 1.21.X though :) ? Should I also update the version number to the latest one (1.21.1) ? Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/t/test-gitmw-lib.sh | 19 ++- contrib/mw-to-git/t/test.config | 3 ++- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/contrib/mw-to-git/t/test-gitmw-lib.sh b/contrib/mw-to-git/t/test-gitmw-lib.sh index 3b2cfac..bb76cee 100755 --- a/contrib/mw-to-git/t/test-gitmw-lib.sh +++ b/contrib/mw-to-git/t/test-gitmw-lib.sh @@ -336,20 +336,21 @@ wiki_install () { fi # Fetch MediaWiki's archive if not already present in the TMP directory + MW_FILENAME="mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz" cd "$TMP" - if [ ! -f "$MW_VERSION.tar.gz" ] ; then - echo "Downloading $MW_VERSION sources ..." - wget "http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0.tar.gz"; || + if [ ! -f $MW_FILENAME ] ; then + echo "Downloading $MW_VERSION_MAJOR.$MW_VERSION_MINOR sources ..." + wget "http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/$MW_FILENAME"; || error "Unable to download "\ - "http://download.wikimedia.org/mediawiki/1.19/"\ - "mediawiki-1.19.0.tar.gz. "\ + "http://download.wikimedia.org/mediawiki/$MW_VERSION_MAJOR/"\ + "$MW_FILENAME. "\ "Please fix your connection and launch the script again." - echo "$MW_VERSION.tar.gz downloaded in `pwd`. "\ + echo "$MW_FILENAME downloaded in `pwd`. "\ "You can delete it later if you want." else - echo "Reusing existing $MW_VERSION.tar.gz downloaded in `pwd`." + echo "Reusing existing $MW_FILENAME downloaded in `pwd`." fi - archive_abs_path=$(pwd)/"$MW_VERSION.tar.gz" + archive_abs_path=$(pwd)/$MW_FILENAME cd "$WIKI_DIR_INST/$WIKI_DIR_NAME/" || error "can't cd to $WIKI_DIR_INST/$WIKI_DIR_NAME/" tar xzf "$archive_abs_path" --strip-components=1 || @@ -431,5 +432,5 @@ wiki_delete () { # Delete the wiki's SQLite database rm -f "$TMP/$DB_FILE" || error "Database $TMP/$DB_FILE could not be deleted." rm -f "$FILES_FOLDER/$DB_FILE" - rm -rf "$TMP/$MW_VERSION" + rm -rf "$TMP/mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz" } diff --git a/contrib/mw-to-git/t/test.config b/contrib/mw-to-git/t/test.config index 958b37b..f835dcc 100644 --- a/contrib/mw-to-git/t/test.config +++ b/contrib/mw-to-git/t/test.config @@ -30,6 +30,7 @@ WEB_WWW=$WEB/www # The variables below are used by the script to install a wiki. # You should not modify these unless you are modifying the script itself. -MW_VERSION=mediawiki-1.19.0 +MW_VERSION_MAJOR=1.20 +MW_VERSION_MINOR=0 FILES_FOLDER=install-wiki DB_INSTALL_SCRIPT=db_install.php -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC 4/4] git-mw: Adding preview tool in git-mw.perl
From: Benoit Person This final commit adds the preview subcommand to git mw. It works as such: 1- Find the remote name of the current branch's upstream and check if it's a mediawiki one. 1b- If it's not found or if it's not a mediawiki one. It will list all the mediawiki remotes configured and ask the user to replay the command with the --remote option set. 2- Parse the content of the local file (or blob) (given as argument) using the distant mediawiki's API 3- Retrieve the current page on the distant mediawiki 4- Replaces all content in that page with the newly parsed one 5- Convert relative links into absolute 6- Save the result on disk The command accepts those options: --autoload | -a tries to launch the newly generated file in the user's default browser (using git web--browse) --remote | -r provides a way to select the distant mediawiki in which the user wants to preview his file (or blob) --output | -o enables the user to choose the output filename. Default output filename is based on the input filename in which the extension '.mw' is replaced with '.html' --blob | -b tells the script that the last argument is a blob and not a filename Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/git-mw.perl | 203 +- 1 file changed, 202 insertions(+), 1 deletion(-) diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl index a2f0aa1..79c6cd0 100644 --- a/contrib/mw-to-git/git-mw.perl +++ b/contrib/mw-to-git/git-mw.perl @@ -12,10 +12,37 @@ use strict; use warnings; use Getopt::Long; +use URI::URL qw(url); +use IO::File; +use LWP::Simple; +use HTML::TreeBuilder; + +use Git; +use MediaWiki::API; +use GitMediawiki qw(smudge_filename connect_maybe); + +#preview parameters +my $file_name; +my $remote_name = ''; +my $preview_file_name = ''; +my $autoload = 0; +my $blob = 0; +sub file { + $file_name = shift; + return $file_name; +} my %commands = ( 'help' => - [\&help, {}, \&help] + [\&help, {}, \&help], + 'preview' => + [\&preview, { + '<>' => \&file, + 'output|o=s' => \$preview_file_name, + 'remote|r=s' => \$remote_name, + 'autoload|a' => \$autoload, + 'blob|b' => \$blob + }, \&preview_help] ); # Search for sub-command @@ -33,6 +60,179 @@ GetOptions( %{$cmd->[1]}, # Launch command &{$cmd->[0]}; +# Preview Functions + +sub preview_help { + print <<'END'; +usage: git mw preview [--remote|-r ] [--autoload|-a] + [--output|-o ] + +-r, --remoteSpecify which mediawiki should be used +-o, --outputName of the output file +-a, --autoload Autoload the page in your default web browser +END + exit; +} + +sub preview { + my $wiki; + my ($remote_url, $wiki_page_name); + my ($content, $content_tree, $template, $html_tree, $mw_content_text); + my $file_content; + + # file_name argumeent is mandatory + if (! defined $file_name) { + die "File not set, see `git mw help` \n"; + } + + if ($blob) { # blob mode + $blob = $file_name; + if ($blob =~ /(.+):(.+)/) { + $file_name = $2; + } + } else { # file mode + if (! -e $file_name) { + die "File $file_name does not exists \n"; + } + } + + # Default preview_file_name is file_name with .html ext + if ($preview_file_name eq '') { + $preview_file_name = $file_name; + $preview_file_name =~ s/\.[^.]+$/.html/; + } + + # Transform file_name into a mediawiki page name + $wiki_page_name = smudge_filename($file_name); + $wiki_page_name =~ s/\.[^.]+$//; + + if ($remote_name eq '') { + # Search current branch upstream branch remote + $remote_name = git_cmd_try { + my $current_branch = + Git::command_oneline('symbolic-ref', '--short', 'HEAD'); + Git::config("branch.$current_branch.remote") } + "%s failed w/ code %d"; + + if ($remote_name) { + $remote_url = mediawiki_remote_url_maybe($remote_name); + } + + # Search all possibles mediawiki r
[PATCH/RFC 3/4] git-mw: Adding git-mw.perl script
From: Benoit Person This script will be used for all tools and command related to a mediawiki remote. In this commit we introduce the tool, the way it parses argument and subcommands and an example of subcommand: "help". It also updates the Makefile so that the new tool is installed properly. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 7 --- contrib/mw-to-git/git-mw.perl | 46 +++ 2 files changed, 50 insertions(+), 3 deletions(-) create mode 100644 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index fe30e7f..c0633b1 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -6,6 +6,7 @@ GIT_MEDIAWIKI_PM=GitMediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL+=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ @@ -19,14 +20,14 @@ install_pm: cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR) build: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ build-perl-script install: install_pm - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ install-perl-script clean: - $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \ clean-perl-script rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \ No newline at end of file diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100644 index 000..a2f0aa1 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,46 @@ +#!/usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Set of tools for git repo with a mediawiki remote. +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Getopt::Long; + +my %commands = ( + 'help' => + [\&help, {}, \&help] +); + +# Search for sub-command +my $cmd = $commands{'help'}; +for (my $i = 0; $i < @ARGV; $i++) { + if (defined $commands{$ARGV[$i]}) { + $cmd = $commands{$ARGV[$i]}; + splice @ARGV, $i, 1; + last; + } +}; +GetOptions( %{$cmd->[1]}, + 'help|h' => \&{$cmd->[2]}); + +# Launch command +&{$cmd->[0]}; + +## Help Functions ## + +sub help { + print <<'END'; +usage: git mw + +git mw commands are: +HelpDisplay help information about git mw +END + exit; +} \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC 1/4] git-mw: Introduction of GitMediawiki.pm
From: Benoit Person The GitMediawiki.pm goal is to share code betwwen several scripts in Git-Mediawiki (for now, git-mw.perl introduced in this patch serie and git-remote-mediawiki.perl) Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/GitMediawiki.pm | 24 contrib/mw-to-git/Makefile| 19 +-- 2 files changed, 41 insertions(+), 2 deletions(-) create mode 100644 contrib/mw-to-git/GitMediawiki.pm diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm new file mode 100644 index 000..8a0ffc7 --- /dev/null +++ b/contrib/mw-to-git/GitMediawiki.pm @@ -0,0 +1,24 @@ +package GitMediawiki; + +use 5.008; +use strict; +use Git; + +BEGIN { + +our ($VERSION, @ISA, @EXPORT, @EXPORT_OK); + +# Totally unstable API. +$VERSION = '0.01'; + +require Exporter; + +@ISA = qw(Exporter); + +@EXPORT = (); + +# Methods which can be called as standalone functions as well: +@EXPORT_OK = (); +} + +1; # Famous last words \ No newline at end of file diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index f149719..fe30e7f 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -4,14 +4,29 @@ # ## Build git-remote-mediawiki +GIT_MEDIAWIKI_PM=GitMediawiki.pm SCRIPT_PERL=git-remote-mediawiki.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \ +-s --no-print-directory instlibdir) all: build -build install clean: +install_pm: + cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR) + +build: + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +build-perl-script + +install: install_pm + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ +install-perl-script + +clean: $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ -$@-perl-script +clean-perl-script + rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \ No newline at end of file -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC 2/4] git-mw: Moving some functions from git-remote-mediawiki.perl to GitMediawiki.pm
From: Benoit Person Moving mediawiki_clean_filename, mediawiki_smudge_filename and mw_connect_maybe Since we have a clean namespace in a perl module, we also rename them into more concise ones (clean_filename, smudge_filename and connect_maybe). It also delete the side effects of mw_connect_maybe requiring it to be called with parameters and returning the new instance of the Mediawiki::Api if it needs to be created Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/GitMediawiki.pm | 72 +- contrib/mw-to-git/git-remote-mediawiki.perl | 80 + 2 files changed, 84 insertions(+), 68 deletions(-) diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm index 8a0ffc7..acf4e43 100644 --- a/contrib/mw-to-git/GitMediawiki.pm +++ b/contrib/mw-to-git/GitMediawiki.pm @@ -18,7 +18,77 @@ require Exporter; @EXPORT = (); # Methods which can be called as standalone functions as well: -@EXPORT_OK = (); +@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe); +} + +# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced +use constant SLASH_REPLACEMENT => '%2F'; + +sub clean_filename { + my $filename = shift; + $filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g; + # [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded. + # Do a variant of URL-encoding, i.e. looks like URL-encoding, + # but with _ added to prevent MediaWiki from thinking this is + # an actual special character. + $filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge; + # If we use the uri escape before + # we should unescape here, before anything + + return $filename; +} + +sub smudge_filename { + my $filename = shift; + $filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g; + $filename =~ s/ /_/g; + # Decode forbidden characters encoded in clean_filename + $filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge; + return $filename; +} + +sub connect_maybe { + my $wiki = shift; + if ($wiki) { + return $wiki; + } + + my $remote_name = shift; + my $remote_url = shift; + my ($wiki_login, $wiki_password, $wiki_domain); + + git_cmd_try { + $wiki_login = Git::config("remote.${remote_name}.mwLogin"); + $wiki_password = Git::config("remote.${remote_name}.mwPassword"); + $wiki_domain = Git::config("remote.${remote_name}.mwDomain");} + "%s failed w/ code %d"; + + $wiki = MediaWiki::API->new; + $wiki->{config}->{api_url} = "${remote_url}/api.php"; + if ($wiki_login) { + my %credential = ( + 'url' => $remote_url, + 'username' => $wiki_login, + 'password' => $wiki_password + ); + Git::credential(\%credential); + my $request = {lgname => $credential{username}, + lgpassword => $credential{password}, + lgdomain => $wiki_domain}; + if ($wiki->login($request)) { + Git::credential(\%credential, 'approve'); + print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n); + } else { + print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n); + print {*STDERR} ' (error ' . + $wiki->{error}->{code} . ': ' . + $wiki->{error}->{details} . ")\n"; + Git::credential(\%credential, 'reject'); + exit 1; + } + } + + return $wiki; } 1; # Famous last words \ No newline at end of file diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index be17e89..d679035 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -13,6 +13,8 @@ use strict; use MediaWiki::API; +use Git; +use GitMediawiki qw(clean_filename smudge_filename connect_maybe); use DateTime::Format::ISO8601; # By default, use UTF-8 to communicate with Git and the user @@ -24,9 +26,6 @@ use IPC::Open2; use warnings; -# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced -use constant SLASH_REPLACEMENT => "%2F"; - # It's not always possible to delete pages (may require some # privileges). Deleted pages are repla
[PATCH/RFC V2 0/4] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. This serie is a second attempt to achieve it: - It adds a new GitMediawiki.pm package to share code between the new tool and `git-remote-mediawiki.perl`. (PATCH 1 & 2) - It creates a new "meta"-command `git mw` with subcommand handling (PATCH 3) - It adds a new subcommand named `preview` to `git mw` (PATCH 4) changes from the V0: - add new package GitMediawiki - move some of git-remote-mediawiki functions into the package - update git-remote-mediawiki to use those "moved" functions - add a hacky-way to install it in the Makefile - use it in the new git mw tool - add a way to give to the preview tool blobs as argument - add a fallback when the upstream's branch remote is not a mediawiki remote - update the `autoload` option to use `git web--browse` and not `xdg-open` - update the way we find the upstream's branch remote name For now, this PATCH/RFC is based on the 'next' branch merged with the bp/mediawiki-credential patch. For the final version, I will try to rebase it on celestin's work with perlcritic. [1] https://github.com/moy/Git-Mediawiki/issues/7 Benoit Person (4): git-mw: Introduction of GitMediawiki.pm git-mw: Moving some functions from git-remote-mediawiki.perl to GitMediawiki.pm git-mw: Adding git-mw.perl script git-mw: Adding preview tool in git-mw.perl contrib/mw-to-git/GitMediawiki.pm | 94 +++ contrib/mw-to-git/Makefile | 22 ++- contrib/mw-to-git/git-mw.perl | 247 contrib/mw-to-git/git-remote-mediawiki.perl | 80 ++--- 4 files changed, 373 insertions(+), 70 deletions(-) create mode 100644 contrib/mw-to-git/GitMediawiki.pm create mode 100644 contrib/mw-to-git/git-mw.perl -- 1.8.3.GIT -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH/RFC] git-remote-mediawiki: new tool to preview local changes without pushing
From: Benoit Person The #7 issue on git-mediawiki's issue tracker [1] states that the ability to preview content without pushing would be a nice thing to have. This commit is a first attempt to achieve it. It adds a new git command, named `git mw`. This command accepts the subcommands `help` and `preview` for now. The default behaviour for the `preview` subcommand is: 1- Find the remote name of the current branch's upstream and check if it's a wiki one with its url (ie: mediawiki://) 2- Parse the content of the local file (given as argument) using the distant wiki's API. 3- Retrieve the current page on the distant mediawiki. 4- Merge those those contents. 5- Convert relative links to absolute ones. 6- Save the result on disk. The command also accepts argument for more controls: --autoload | -a tries to launch the newly generated file in the user's default browser --remote | -r provides a way to select the distant mediawiki in which the user wants to preview his file. --output | -o enables the user to choose the output filename. Default output filename is based on the input filename in which the extension `.mw` is replaced with `.html`. It works but a couple of points trouble me: 1- I had to copy two functions from `git-remote-mediawiki.perl`, I don't really know how we could "factorize" those things ? I don't think it makes much sense to create a package just for them ? 2- The current behavior is to crash if the current branch do not have an upstream branch on a valid mediawiki remote. To find that specific remote, it runs `git rev-parse --symbolic-full-name @{upstream}` which will return something like `/refs/remotes/$remote_name/master`. 2a- maybe there is a better way to find that remote name ? 2b- would it be useful to add a fallback if that search fails ? searching for a valid mediawiki remote url in all the remotes returned by `git remote` for instance ? 3- The current idea of merging the content of the mediawiki's current page with new content fails when the page is a local creation. (Hence the error message when we get a bad result from the `get` call l.129). I thought about two ways to overcome this: 3a- Use the wiki's homepage for the template. 3b- A two-step process: - first we create a general template (based on the wiki's homepage) with a specific flag at the content position. This step is done only if that template do not exists locally. - replace the specific flag with the newly parsed content. The downfall of those two "solutions" is on links like 'talk', 'view source' etc ... those will need to be updated to. And evven then, it will still fails for page created only locally. 4- In the Makefile, there is certainly a more "Makefily" way to do it but I had no luck finding it and still preserving the factorization for the `build`, `install` and `clean` target. I ended up with something like this: build: $(BUILD_SCRIPTS) $(BUILD_SCRIPTS): $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$@ \ build-perl-script install: ... clean: ... but clearly, for only two scripts, it's more a refucktoring than a refactoring :/ . [1] https://github.com/moy/Git-Mediawiki/issues/7 Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/Makefile| 4 + contrib/mw-to-git/git-mw.perl | 249 ++ 2 files changed, 253 insertions(+) create mode 100644 contrib/mw-to-git/git-mw.perl diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile index f149719..0cba1e3 100644 --- a/contrib/mw-to-git/Makefile +++ b/contrib/mw-to-git/Makefile @@ -5,13 +5,17 @@ ## Build git-remote-mediawiki SCRIPT_PERL=git-remote-mediawiki.perl +SCRIPT_PERL_MW=git-mw.perl GIT_ROOT_DIR=../.. HERE=contrib/mw-to-git/ SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL)) +SCRIPT_PERL_MW_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL_MW)) all: build build install clean: $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \ $@-perl-script + $(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_MW_FULL) \ +$@-perl-script \ No newline at end of file diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl new file mode 100644 index 000..a92c459 --- /dev/null +++ b/contrib/mw-to-git/git-mw.perl @@ -0,0 +1,249 @@ +#! /usr/bin/perl + +# Copyright (C) 2013 +# Benoit Person +# Celestin Matte +# License: GPL v2 or later + +# Tools attached to git-remote-mediawiki (help, preview ...) +# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/ + +use strict; +use warnings; + +use Git; +use MediaWiki::API; +
[PATCH] git-remote-mediawiki: use Git.pm functions for credentials
From: Benoit Person In 52dce6d, a new credential function was added to Git.pm, based on git-remote-mediawiki's functions. The logical follow-up is to use those functions in git-remote-mediawiki. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/git-remote-mediawiki.perl | 66 - 1 file changed, 9 insertions(+), 57 deletions(-) diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 9c14c1f..6672e4c 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -13,6 +13,7 @@ use strict; use MediaWiki::API; +use Git; use DateTime::Format::ISO8601; # By default, use UTF-8 to communicate with Git and the user @@ -156,57 +157,6 @@ while () { ## Functions ## -## credential API management (generic functions) - -sub credential_read { - my %credential; - my $reader = shift; - my $op = shift; - while (<$reader>) { - my ($key, $value) = /([^=]*)=(.*)/; - if (not defined $key) { - die "ERROR receiving response from git credential $op:\n$_\n"; - } - $credential{$key} = $value; - } - return %credential; -} - -sub credential_write { - my $credential = shift; - my $writer = shift; - # url overwrites other fields, so it must come first - print $writer "url=$credential->{url}\n" if exists $credential->{url}; - while (my ($key, $value) = each(%$credential) ) { - if (length $value && $key ne 'url') { - print $writer "$key=$value\n"; - } - } -} - -sub credential_run { - my $op = shift; - my $credential = shift; - my $pid = open2(my $reader, my $writer, "git credential $op"); - credential_write($credential, $writer); - print $writer "\n"; - close($writer); - - if ($op eq "fill") { - %$credential = credential_read($reader, $op); - } else { - if (<$reader>) { - die "ERROR while running git credential $op:\n$_"; - } - } - close($reader); - waitpid($pid, 0); - my $child_exit_status = $? >> 8; - if ($child_exit_status != 0) { - die "'git credential $op' failed with code $child_exit_status."; - } -} - # MediaWiki API instance, created lazily. my $mediawiki; @@ -217,22 +167,24 @@ sub mw_connect_maybe { $mediawiki = MediaWiki::API->new; $mediawiki->{config}->{api_url} = "$url/api.php"; if ($wiki_login) { - my %credential = (url => $url); - $credential{username} = $wiki_login; - $credential{password} = $wiki_passwd; - credential_run("fill", \%credential); + my %credential = ( + 'url' => $url, + 'username' => $wiki_login, + 'password' => $wiki_passwd + ); + Git::credential(\%credential); my $request = {lgname => $credential{username}, lgpassword => $credential{password}, lgdomain => $wiki_domain}; if ($mediawiki->login($request)) { - credential_run("approve", \%credential); + Git::credential(\%credential, 'approve'); print STDERR "Logged in mediawiki user \"$credential{username}\".\n"; } else { print STDERR "Failed to log in mediawiki user \"$credential{username}\" on $url\n"; print STDERR " (error " . $mediawiki->{error}->{code} . ': ' . $mediawiki->{error}->{details} . ")\n"; - credential_run("reject", \%credential); + Git::credential(\%credential, 'reject'); exit 1; } } -- 1.8.3.rc3.7.gc2f33ed.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] git-remote-mediawiki: use git.pm functions for credentials
From: Benoit Person In 52dce6d, a new credential function was added to git.pm, based on git-remote-mediawiki's functions. The logical follow-up is to use those functions in git-remote-mediawiki. Signed-off-by: Benoit Person Signed-off-by: Matthieu Moy --- contrib/mw-to-git/git-remote-mediawiki.perl | 66 - 1 file changed, 9 insertions(+), 57 deletions(-) diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl index 9c14c1f..9fb281e 100755 --- a/contrib/mw-to-git/git-remote-mediawiki.perl +++ b/contrib/mw-to-git/git-remote-mediawiki.perl @@ -13,6 +13,7 @@ use strict; use MediaWiki::API; +use Git; use DateTime::Format::ISO8601; # By default, use UTF-8 to communicate with Git and the user @@ -156,57 +157,6 @@ while () { ## Functions ## -## credential API management (generic functions) - -sub credential_read { - my %credential; - my $reader = shift; - my $op = shift; - while (<$reader>) { - my ($key, $value) = /([^=]*)=(.*)/; - if (not defined $key) { - die "ERROR receiving response from git credential $op:\n$_\n"; - } - $credential{$key} = $value; - } - return %credential; -} - -sub credential_write { - my $credential = shift; - my $writer = shift; - # url overwrites other fields, so it must come first - print $writer "url=$credential->{url}\n" if exists $credential->{url}; - while (my ($key, $value) = each(%$credential) ) { - if (length $value && $key ne 'url') { - print $writer "$key=$value\n"; - } - } -} - -sub credential_run { - my $op = shift; - my $credential = shift; - my $pid = open2(my $reader, my $writer, "git credential $op"); - credential_write($credential, $writer); - print $writer "\n"; - close($writer); - - if ($op eq "fill") { - %$credential = credential_read($reader, $op); - } else { - if (<$reader>) { - die "ERROR while running git credential $op:\n$_"; - } - } - close($reader); - waitpid($pid, 0); - my $child_exit_status = $? >> 8; - if ($child_exit_status != 0) { - die "'git credential $op' failed with code $child_exit_status."; - } -} - # MediaWiki API instance, created lazily. my $mediawiki; @@ -217,22 +167,24 @@ sub mw_connect_maybe { $mediawiki = MediaWiki::API->new; $mediawiki->{config}->{api_url} = "$url/api.php"; if ($wiki_login) { - my %credential = (url => $url); - $credential{username} = $wiki_login; - $credential{password} = $wiki_passwd; - credential_run("fill", \%credential); + my %credential = ( + 'url' => $url, + 'username' => $wiki_login, + 'password' => $wiki_passwd + ); + Git::credential \%credential; my $request = {lgname => $credential{username}, lgpassword => $credential{password}, lgdomain => $wiki_domain}; if ($mediawiki->login($request)) { - credential_run("approve", \%credential); + Git::credential \%credential, 'approve'; print STDERR "Logged in mediawiki user \"$credential{username}\".\n"; } else { print STDERR "Failed to log in mediawiki user \"$credential{username}\" on $url\n"; print STDERR " (error " . $mediawiki->{error}->{code} . ': ' . $mediawiki->{error}->{details} . ")\n"; - credential_run("reject", \%credential); + Git::credential \%credential, 'reject'; exit 1; } } -- 1.8.3.rc3.7.gc2f33ed.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html