This and other RFCs are available on the web at
http://dev.perl.org/rfc/
=head1 TITLE
Replace C<format> built-in with pragmatically-induced C<format> function
=head1 VERSION
Maintainer: Damian Conway <[EMAIL PROTECTED]>
Date: 15 September 2000
Mailing List: [EMAIL PROTECTED]
Number: 230
Version: 1
Status: Developing
=head1 ABSTRACT
This RFC proposes that Perl's existing C<format> mechanism be replaced
with a standard module based on parts of the Text::Autoformat module.
=head1 DESCRIPTION
I can never remember how formats work. The specification syntax is confusing
to me. And I usually don't want the formatted text going straight down some
magical output stream.
It all came to a head when I was building Text::Autoformat. The smart text
recognition was easy -- trying to do the final formatting with C<formline>
and $^A was just too painful.
So I created the C<Text::Autoformat::form> subroutine. It uses a template
specification than fits my brain better, it's deeply DWIMical, it's highly
configurable, and it's re-entrant (so you can use a C<form> to format
another C<form>'s headers and footers, for example).
I propose that the existing C<format> mechanism be removed from Perl 6
and be replaced with a pragma-induced add-in function, based on
the semantics of C<Text::Autoformat::form>, as described in
the following sections.
=head2 The C<format::format> function
The function itself would be called C<format> and would be
imported with a C<use format> pragma.
It takes a format (or "picture" or "template") string followed by one or
more replacement values. It then interpolates those values into each picture
string, and either:
=over 4
=item *
returns the result as a single multi-line string (in a scalar context)
=item *
returns the result as a list of single-line strings (in a list context)
=item *
prints the result to the current filehandle (in a void context).
=back
A picture string consists of sequences of the following characters:
=over 4
=item <<<<<<<<
Left-justified field indicator. A series of two or more sequential <'s
specify a left-justified field to be filled by a subsequent value.
=item >>>>>>>>
Right-justified field indicator. A series of two or more sequential >'s
specify a right-justified field to be filled by a subsequent value.
=item ^^^^^^^^
Centre-justified field indicator. A series of two or more sequential ^'s
specify a centred field to be filled by a subsequent value.
=item >>>.<<<<
A numerically formatted field with the specified number of digits to
either side of the decimal place. See L<Numerical formatting> below.
=item [[[[[[[[
Left-justified block field indicator.
Just like a <<<<<<<<< field, except it repeats as required on subsequent
lines. See below.
=item ]]]]]]]]
Right-justified block field indicator.
Just like a >>>>>>>> field, except it repeats as required on subsequent
lines. See below.
=item ||||||||
Centre-justified block field indicator.
Just like a ^^^^^^^^ field, except it repeats as required on subsequent
lines. See below.
=item ]]].[[[[
A numerically formatted block field with the specified number of digits to
either side of the decimal place.
Just like a >>>.<<<< field, except it repeats as required on
subsequent lines. See below.
=item ~
A single character field. Interpolates a single character from a subsequent
data value. Repeats on subsequent lines as required.
=item \
Literal escape of next character (e.g. C<\~> is formatted as a literal '~',
not a one character wide field).
=item Any other character
That literal character.
=back
Hence a typical use of C<format> might look like this:
$formatted = format "<[[[[[[[[[[[[[[[[> >>>>>>>>>>>>>>>>",
$aphorism, "page $page_num";
and might produce an output like this:
<Like a camel > page 123
<through the eye >
<of a needle, so >
<are the days of >
<our lives >
Note that, because every field (except a C<~> field) must be at least
two characters wide, the single C<E<lt>> and C<E<gt>> brackets in the
format string are correctly interpreted as literals.
=head3 Multi-line filling
As the previous example indicates, any line with a block field continues
interpolation of that field on subsequent lines until all block fields
in the format have consumed all their data. Non-block fields on these
lines are replaced by the appropriate number of spaces.
For example:
$title = "On The Evil That Is Spam";
$text1 = "How many times have you longed to smash...";
$text2 = "...the bedevilment that is spam?";
format "<<<<<<<<<<< [[[[[[[[[[[[[[[ [[[[[[[[[[",
$title, $text1, $text2;
# prints:
#
# On The Evil How many times ...the be-
# have you longed devilment
# to smash... that is
# spam
=head3 Hyphenation
As the previous example indicates, within a block field, words are
wrapped whole, unless they will not fit into the field at all, in which
case they are broken and hyphenated. Simple hyphenation is
used (i.e. break at the I<N-1>th character and insert a '-'), unless a
suitable alternative subroutine is specified instead.
Words will not be broken if the break would leave less than 2 characters
of the word on the current line. This minimum can be varied by setting the
'minbreak' option to some positive numeric value. This value indicates
the minumum total broken characters (I<including> hyphens) that must be
left on the current line.
Alternative word breaking subroutines can be specified using the "break"
option in a configuration hash. For example:
format { break => \&my_line_breaker },
$template,
@data;
C<format> expects any user-defined line-breaking subroutine to take three
arguments (the string to be broken, the maximum permissible length of
the initial section, and the total width of the field being filled).
The subroutine must return a list of two strings: the initial
(broken) section of the word, and the remainder of the string
respectively).
For example:
sub tilde_break = sub($$$)
{
(substr($_[0],0,$_[1]-1).'~', substr($_[0],$_[1]-1));
}
format { break => \&tilde_break },
$template,
@data;
This causes '~' to be used as the hyphenation character.
The C<use format> pragma provides two exportable functions to
simplify the use of variant hyphenation schemes.
The exportable function C<format::break_with> takes a single string
argument and returns a reference to a sub which hyphenates with that
string. Hence the previous example could be rewritten:
use format qw( break_wrap );
format { break => break_with('~') },
$template,
@data;
The exportable function C<format::break_TeX> takes an optional argument
specifying a TeX hyphenation file and returns a reference to a sub which
hyphenates using Jan Pazdziora's TeX::Hyphen module (assuming it is
installed). For example:
use format qw( break_TeX );
format { break => break_TeX("lithuanian.hy") }
$template,
@data;
Note that in the previous examples there is no leading '\&' before
C<break_with> or C<break_TeX>, since each is being directly I<called>,
arther than referred to. They each return a reference to some other
suitable word-breaking subroutine.
=head3 Controlling line filling
C<format> preserves the original whitespace (including newlines) from an
interpolated string, unless called with certain options.
The "squeeze" option (when specified with a true value) causes any
interpolated sequence of spaces and/or tabs (but not newlines) to be
replaced with a single space.
The "fill" option independently causes newlines to be "squeezed".
Hence:
$frmt = "# [[[[[[[[[[[[[[[[[[[[[";
$data = "h e\t \tl lo\nworld\t\t\t\t\t";
format $frmt, $data;
# h e l lo
# world
format {squeeze=>1}, $frmt, $data;
# h e l lo
# world
format {fill=>1}, $frmt, $data;
# h e l lo world
format {squeeze=>1, fill=>1}, $frmt, $data;
# h e l lo world
Whether or not filling or squeezing is in effect, C<format> can also be
directed to trim any extra whitespace from the I<end> of each line it
formats, using the "trim" option. If this option is specified with a
true value, every line returned by C<format> will automatically have the
substitution C<s/[ \t]+$//gm> applied to it.
Hence:
print length format "[[[[[[[[[[", "short";
# 11
print length format {trim=>1}, "[[[[[[[[[[", "short";
# 6
=head3 How C<format> consumes strings
Within a single call to C<format> fields consume data
text as they format it, so the following:
$text = "a line of text to be formatted over three lines";
format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
produces:
a line of
text to
be fo-
not:
a line of
a line
a line
To achieve the latter effect, the variable arguments must be converted
to independent literals (by double-quoted interpolation):
format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
"$text", "$text", "$text";
Although values passed from variable arguments are progressively consumed
I<within> C<format>, the values of the original variables passed to C<format>
are I<not> altered. In other words, internally C<format> operates on a
copy of each variable's string. Hence:
format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
print $text, "\n";
will print:
a line of
text to
be fo-
a line of text to be formatted over three lines
To cause C<format> to I<visibly> consume the values of the original
variables passed to it, they must be passed as references. Thus:
format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
\$text, \$text, \$text;
print $text, "\n";
will print:
a line of
text to
be fo-
rmatted over three lines
Note that, for safety, the "non-consuming" behaviour takes precedence,
so if a variable is passed to C<format> both by reference I<and> by value,
its final value will be unchanged.
=head3 Numerical formatting
The ">>>.<<<" and "]]].[[[" field specifiers may be used to format
numeric values about a fixed decimal place marker. For example:
format '(>>>>>.<<)', <<EONUMS;
1
1.0
1.001
1.009
123.456
1234567
one two
EONUMS
would print:
( 1.0 )
( 1.0 )
( 1.00)
( 1.01)
( 123.46)
(#####.##)
(?????.??)
(?????.??)
Fractions are rounded (a la C<sprintf>) to the specified number of
places after the decimal, but only the minimal number of significant
digits are shown. That's why, in the above example, 1 and 1.0 are
formatted as "1.0", whilst 1.001 is formatted as "1.00".
It is possible to specify that the I<maximal> number of decimal places
always be used, by giving the configuration option 'numeric' the value
'AllPlaces'. For example:
format { numeric => 'AllPlaces' },
'(>>>>>.<<)', <<'EONUMS';
1
1.0
EONUMS
would print:
( 1.00)
( 1.00)
Note that although decimal digits are rounded to fit the specified width, the
integral part of a number is never modified. If there are not enough places
before the decimal place to represent the number, the entire number is
replaced with hashes.
If a I<non-numeric> sequence is passed as data for a numeric field, it is
formatted as a series of question marks. This querulous behaviour can be
changed by giving the configuration option 'numeric' the value 'SkipNaN',
in which case, any invalid numeric data is simply ignored. For example:
format { numeric => 'SkipNaN' },
'(>>>>>.<<)',
<<EONUMS;
1
two three
4
EONUMS
would print:
( 1.0 )
( 4.0 )
=head3 Filling block fields with lists of values
If the data provided for a particular field is an array reference,
then C<format> automatically joins the elements of the array into a single
string, separating each element with a newline character. As a result, a
call like this:
@values = qw( 1 10 100 1000 );
format "(]]]].[[)", \@values;
will print out
( 1.00)
( 10.00)
( 100.00)
(1000.00)
as might be expected.
Note that, because arrays must be passed using a reference, their
original contents are consumed by C<format>, just like the contents of
scalars passed by reference.
To avoid having an array consumed by C<format>, pass it as an anonymous
array:
format "(]]]].[[)", [@values];
=head3 Headers, footers, and pages
The C<format> subroutine can also insert headers, footers, and page-feeds
as it formats. These features are controlled by the "header", "footer",
"pagefeed", and "pagelen" options.
The "pagelen" option specifies the total number of lines in a page (including
headers and footers, but I<not> page-feeds).
If the "header" option is specified with a string value, that string is
used as the header of every page generated. If it is specified as a reference
to a subroutine, that subroutine is called at the start of every page and
its return value used as the header string. When called, the subroutine is
passed the current page number.
Likewise, if the "footer" option is specified with a string value, that
string is used as the footer of every page generated. If it is specified
as a reference to a subroutine, that subroutine is called at the I<start>
of every page and its return value used as the footer string. When called,
the footer subroutine is passed the current page number.
The "pagefeed" option acts in exactly the same way, to produce a
pagefeed which is appended after the footer. But note that the pagefeed
is not counted as part of the page length.
All three of these page components are recomputed at the start of each
new page, before the page contents are formatted (recomputing the header
and footer makes it possible to determine how many lines of data to
format so as to adhere to the specified page length).
When the call to C<format> is complete and the data has been fully formatted,
the footer subroutine is called one last time, with an extra argument of 1.
The string returned by this final call is used as the final footer.
So for example, a series of 60-line pages with appropriate headers and
footers might be formatted like so:
format { header => sub { "Page $_[0]\n\n" },
footer => sub { return "" if $_[1];
"-"x50."\n".format ">"x50", "...".($_[0]+1);
},
pagefeed => "\n\n",
pagelen => 60
},
$template,
@data;
Note the recursive use of C<format> within the "footer" option.
=head3 Lexically permanent options
If C<use format> is called with a hash reference as an argument, the
entries of that hash specify options that are to be made the defaults
until the end of the current lexical scope. For example, to cause
C<format> to always squeeze and trim whitespace but respect newlines,
the following line would be placed at the start of the source file:
use format { squeeze=>1, fill=>0, trim=>1 };
With these defaults in effect, filling could be turned on and trimming disabled
within a particular subroutine like so:
sub particular {
use format { fill => 1, trim => 0 };
# do filled, untrimmed formatting here
} # format's defaults revert to previous values at end of scope
=head3 C<format> examples
As an example of the use of C<format>, the following:
$count = 1;
$text = "A big long piece of text to be formatted exquisitely";
format q{
|||| <<<<<<<<<<
----------------
^^^^ ]]]]]]]]]]\|
=
]]].[[[
}, $count, $text, $count+11, $text, "123 123.4\n123.456789";
produces the following output:
1 A big long
----------------
12 piece of|
text to be|
formatted|
exquisite-|
ly|
=
123.0
123.4
123.456
Alternatively, picture strings and replacement values can be interleaved.
For example:
$report = format
'Name Rank Serial Number',
'==== ==== =============',
'<<<<<<<<<<<<< ^^^^ <<<<<<<<<<<<<',
$name, $rank, $serial_number,
''
'Age Sex Description',
'=== === =====================',
'^^^ ^^^ [[[[[[[[[[[]]]]]]]]]]',
$age, $sex, $description;
=head1 MIGRATION ISSUES
Non-trivial, but do-able.
=head1 IMPLEMENTATION
See Text::Autoformat module.
The proposed built-in/add-in ought to be reimplemented in C for speed.
=head1 REFERENCES
Text::Autoformat
