File Coverage

=head1 NAME

App::makefilepl2cpanfile - Convert Makefile.PL to a cpanfile automatically

=head1 VERSION

This document describes App::makefilepl2cpanfile version 0.06.

=cut

=head1 SYNOPSIS

Create or refresh the F<cpanfile> of the project in the current directory:

        use App::makefilepl2cpanfile;
        use Path::Tiny;

        my $text = App::makefilepl2cpanfile::generate();
        path('cpanfile')->spew_utf8($text);

Keep the hand-written C<develop> section of an existing F<cpanfile>:

        my $old  = path('cpanfile')->slurp_utf8;
        my $text = App::makefilepl2cpanfile::generate(existing => $old);
        path('cpanfile')->spew_utf8($text);

Make a F<cpanfile> for another project, with no developer tools added.
The result is the same on every computer, which is useful in CI:

        my $text = App::makefilepl2cpanfile::generate({
                makefile     => '/path/to/other/project/Makefile.PL',
                with_develop => 0,
        });

List every dependency without making a F<cpanfile>:

        my $deps = App::makefilepl2cpanfile::parse_prereqs(
                path('Makefile.PL')->slurp_utf8
        );
        for my $phase (sort keys %{$deps}) {
                for my $rel (sort keys %{ $deps->{$phase} }) {
                        for my $module (sort keys %{ $deps->{$phase}{$rel} }) {
                                print "$phase $rel $module\n";
                        }
                }
        }

Check in a test that the committed F<cpanfile> is up to date:

        use Test::More;
        is(
                App::makefilepl2cpanfile::generate(existing => path('cpanfile')->slurp_utf8),
                path('cpanfile')->slurp_utf8,
                'cpanfile matches Makefile.PL',
        );

From the command line (see L<makefilepl2cpanfile>):

        makefilepl2cpanfile              # write ./cpanfile
        makefilepl2cpanfile --dry-run    # print it instead
        makefilepl2cpanfile --diff       # show what would change

=head1 DESCRIPTION

A Perl distribution lists the modules it needs in its F<Makefile.PL>.
Many tools (for example C<cpanm --installdeps .> and CI systems) prefer
a F<cpanfile> instead.  This module reads a F<Makefile.PL> and writes the
same list in F<cpanfile> form, so you only keep the list in one place.

The F<Makefile.PL> is B<never run>.  It is read as plain text and
searched for the parts that list dependencies.  This makes the tool safe
to use on code you do not trust, but it also means that it cannot see
dependencies that are computed while the program runs (see
L</COMMON PITFALLS>).

Reading takes time in proportion to the size of the input, however it is
written, so a hostile F<Makefile.PL> or F<cpanfile> (for example one with
thousands of unclosed braces) cannot make the tool hang.

=head2 What it reads

=over 4

=item * C<PREREQ_PM>, C<BUILD_REQUIRES>, C<TEST_REQUIRES> and
C<CONFIGURE_REQUIRES>.  These become C<requires> lines in the C<runtime>,
C<build>, C<test> and C<configure> phases.

=item * C<prereqs =E<gt> { PHASE =E<gt> { RELATIONSHIP =E<gt> { ... } } }>
blocks (the CPAN::Meta::Spec version 2 layout), wherever they appear -
also inside C<META_MERGE>.

=item * The older version 1 layout, directly inside C<META_MERGE>:
C<requires>, C<recommends>, C<suggests> and C<conflicts> become lines of
that kind in the C<runtime> phase; C<build_requires> and
C<configure_requires> become C<requires> lines in the C<build> and
C<configure> phases.

=item * C<MIN_PERL_VERSION>, which becomes C<requires 'perl', 'VERSION';>.

=item * A C<#> comment after an entry.  It is copied to the F<cpanfile>.

=back

=head2 What it writes

        # Generated from Makefile.PL using makefilepl2cpanfile

        requires 'perl', '5.010';

        requires 'Moo', '2.000';   # object system
        requires 'Try::Tiny';
        recommends 'JSON::MaybeXS';

        on 'test' => sub {
                requires 'Test::More', '0.98';
        };

        on 'develop' => sub {
                requires 'Perl::Critic';
        };

Runtime dependencies come first, without an C<on> block.  The other
phases follow in this order: C<configure>, C<build>, C<test>, C<develop>.
Inside each phase, C<requires> lines come first, then C<recommends>, then
C<suggests>, and the modules are in alphabetical order.  The same input
always gives exactly the same output.

=head1 CONFIGURATION

When C<with_develop> is true (the default), some developer tools are
added to the C<develop> phase.  By default these are L<Devel::Cover>,
L<Perl::Critic>, L<Test::Pod> and L<Test::Pod::Coverage>.

To choose your own list, create the file
F<~/.config/makefilepl2cpanfile.yml>:

        develop:
          Perl::Critic: 0
          Devel::Cover: 0
          My::Extra::Tool: '1.00'

The number after each name is the minimum version; C<0> means "any
version".  Your list B<replaces> the default list; it is not added to it.
Names that are not valid module names, and versions that are not version
numbers, are ignored with a warning, so a mistake in this file can never
put unexpected code into your F<cpanfile>.

=head1 DATA STRUCTURE

C<parse_prereqs()> returns a hash reference with three levels:

        {
                PHASE => {
                        RELATIONSHIP => {
                                'Module::Name' => { version => '1.0', comment => 'why it is needed' },
                        },
                },
        }

=over 4

=item * PHASE is one of C<runtime>, C<configure>, C<build>, C<test> or
C<develop>.

=item * RELATIONSHIP is one of C<requires>, C<recommends>, C<suggests> or
C<conflicts>.

=item * C<version> is the version requirement exactly as it was written:
a single version (C<'1.60'> stays C<'1.60'>) or a version range such as
C<'E<gt>= 1.2, E<lt> 2.0'>.  It is C<0> when there is no requirement.

=item * C<comment> is the text of the C<#> comment after the entry, or
C<undef> when there is none.

=back

A phase or relationship with no modules is not present at all.

=head1 ENCODING

=over 4

=item * B<The Makefile.PL file> is read as UTF-8.  Comments may contain
any printable Unicode text, including accented letters, emoji and
combining marks; they are copied to the output unchanged.  Control
characters (other than TAB) and Unicode direction-override characters
are removed from comments, so that the generated file cannot be made to
look different from what it really contains.  If the file is not valid UTF-8 you get a
warning and processing continues (see L</generate(%args)>).

=item * B<The returned cpanfile text> is a Perl character string.  Write
it with a UTF-8 output layer, for example C<path(...)-E<gt>spew_utf8> or
C<binmode $fh, ':encoding(UTF-8)'>.  Otherwise Perl warns
"Wide character in print" when a comment contains non-ASCII text.

=item * B<The existing argument> and B<the content argument of
parse_prereqs()> must be character strings: decode them first (for
example with C<slurp_utf8>).  Raw bytes work for plain ASCII, but a
non-ASCII comment would come back as separate bytes.

=item * B<Module names> must be plain ASCII (A-Z, a-z, 0-9, C<_> and
C<::>), as CPAN requires.  A name with any other character is ignored.
This also stops look-alike names, such as C<Test::More> written with a
Cyrillic letter.

=item * B<Version numbers> may use only the ASCII digits 0-9, C<.>,
C<_> and a leading C<v>, and must contain at least one digit.  A version
range joins such numbers, each after one of the operators C<E<gt>=>,
C<E<lt>=>, C<==>, C<!=>, C<E<gt>> or C<E<lt>>, with commas.  The whole
value is checked: C<'1.0-TRIAL'> is not shortened to C<'1.0'>.  Anything
else is treated as "no minimum version".

=item * B<The YAML configuration file> is read as UTF-8; the same rules
for names and versions apply.

=item * B<File names> are passed to the operating system unchanged.
Names with spaces, non-ASCII letters or shell characters such as C<;>
and C<|> are safe: no shell is ever used.

=back

=head1 METHODS

=head2 generate(%args)

=head3 PURPOSE

Reads a F<Makefile.PL> and returns the text of a matching F<cpanfile>.
The F<Makefile.PL> is read as text and is never run.

=head3 ARGUMENTS

Give the arguments as a list of name/value pairs, or as one hash
reference.  All of them are optional.

=over 4

=item * C<content> - the text of a F<Makefile.PL>, for example from
L</read_makefile($path)>.  When it is given it is used as it is, and
C<makefile> is ignored and nothing is read.  Use it to read the file once
and use the same text for more than one purpose.

=item * C<makefile> - the path of the F<Makefile.PL> to read.  Default:
C<'Makefile.PL'> in the current directory.  Anything that turns into a
path when used as a string is accepted, such as a L<Path::Tiny> object.
A filehandle or another kind of reference is refused with
C<Cannot read>.

=item * C<existing> - the text of your current F<cpanfile>.  Default:
C<''> (none).  Only its C<on 'develop' =E<gt> sub { ... }> section is
used.  Every C<requires>, C<recommends>, C<suggests> and C<conflicts>
line in that section is copied to the new text, so your hand-written developer
dependencies are kept.  Lines that are commented out are not copied.
An entry whose name is not a valid module name is dropped.  An entry
whose version is not a version number or version range is kept without a
version, and you get a warning.

=item * C<with_develop> - true or false.  Default: true.  When true, the
developer tools from the configuration file (see L</CONFIGURATION>), or
the default tools, are added to the C<develop> phase as C<requires>.  A
tool that is already in the develop phase, from the F<Makefile.PL> or
from C<existing>, is left exactly as it is.

=back

=head3 RETURNS

A string: the complete F<cpanfile>.  It always starts with the line
C<# Generated from Makefile.PL using makefilepl2cpanfile> and always ends
with exactly one newline.  The layout is described in
L</What it writes>.  A F<Makefile.PL> with no dependencies gives just
the first line.

=head3 SIDE EFFECTS

=over 4

=item * Reads the C<makefile> file.

=item * When C<with_develop> is true, reads
F<~/.config/makefilepl2cpanfile.yml> if it exists.

=item * Never writes, creates or deletes any file.

=item * May print warnings (see MESSAGES below).

=item * Does not change the caller's C<$@>, C<$!> or C<$_>.

=back

=head3 USAGE EXAMPLE

        use App::makefilepl2cpanfile;
        use Path::Tiny;

        my $cpanfile = path('cpanfile');
        my $text = App::makefilepl2cpanfile::generate(
                makefile     => 'Makefile.PL',
                existing     => $cpanfile->exists ? $cpanfile->slurp_utf8 : '',
                with_develop => 1,
        );
        $cpanfile->spew_utf8($text);

=head3 HOW IT WORKS

=over 4

=item 1. Check that C<makefile> is a readable, regular file.

=item 2. Read it as UTF-8.  If that fails because of bad UTF-8, warn and
read the raw bytes instead.  Any other read error is passed on.

=item 3. Find C<MIN_PERL_VERSION> and call L</parse_prereqs($content)>.

=item 4. Copy the develop entries from C<existing>, without replacing
entries that came from the F<Makefile.PL>.

=item 5. If C<with_develop> is true, add each configured tool that is
not already in the develop phase.

=item 6. Format and return the text.

=back

=head3 API SPECIFICATION

=head4 Input

        {
                content => {
                        type     => 'string',
                        optional => 1,
                },
                makefile => {
                        type     => 'string',
                        optional => 1,
                        min      => 1,
                        default  => 'Makefile.PL',
                },
                existing => {
                        type     => 'string',
                        optional => 1,
                        default  => '',
                },
                with_develop => {
                        type     => 'boolean',
                        optional => 1,
                        default  => 1,
                },
        }

=head4 Output

        {
                type    => 'string',
                matches => qr/\A# Generated from Makefile\.PL using makefilepl2cpanfile\n.*(?<!\n)\n\z/s,
        }

=head4 Domains

Each argument, split into groups of values that behave the same way.
"Refused" means the call dies with C<Cannot read '...'>.

        makefile
          valid:    a readable regular file, given as a relative path, an
                    absolute path, or an object that stringifies to one
                    (e.g. Path::Tiny).  Any characters the file system allows,
                    including spaces, non-ASCII letters and shell characters.
          default:  undef or not given -> 'Makefile.PL'
          refused:  '' and '0' (unless such a file exists), a missing file,
                    a directory, a device, a FIFO, a symlink loop, a file
                    without read permission, any reference (filehandle, glob,
                    array, hash, code)
          limits:   a file name component up to the file system's NAME_MAX
                    (usually 255 bytes) is accepted; one byte more is refused.
                    File size: 0 bytes gives just the header line; there is
                    no upper limit other than memory.

        content
          given:    used as the Makefile.PL text; makefile is then ignored
                    and no file is read (so no "Cannot read" is possible)
          default:  undef or not given -> the makefile file is read

        existing
          valid:    any string.  Only the first on 'develop' => sub { ... };
                    block is used; its closing "};" must start a line (after
                    optional spaces or tabs).  'develop' or "develop".
          default:  undef or not given -> ''
          ignored:  a string without a develop block, a block that is never
                    closed, a reference (its "HASH(0x...)" text has no block)
          entries:  0 or more; each needs a valid module name; a version, if
                    given, must be a version or version range (see
                    parse_prereqs)

        with_develop
          true:     any true Perl value, including 'yes' and '0.0'
          false:    0, '0', ''
          default:  undef or not given -> true

        combinations
          - with_develop false does not stop the existing develop block from
            being kept; it only stops tools being added.
          - An empty Makefile.PL with an existing develop block gives the
            header plus that develop block.
          - A configuration file with an empty "develop: {}" adds no tools.

=head3 MESSAGES

Values taken from files or the environment (paths, module names,
versions, parser errors) are shown with control characters and Unicode
direction-override characters replaced by C<\x{..}> escapes, so a
hostile file cannot use a message to send escape sequences to your
terminal.

Errors (the call dies):

        Cannot read '$makefile'
            The path does not exist, is not a regular file (for example a
            directory, a device or a FIFO), cannot be read, or is not a path
            at all (for example a filehandle).
            What to do: check the path and its permissions.

        Failed to parse $cfg_file: $error
            The configuration file exists but contains invalid YAML, cannot be
            read, or its location cannot be checked (for example "Permission
            denied").  A missing file, or something that is not a regular
            file, is simply treated as "no configuration".
            What to do: fix the YAML or the permissions, or delete the file.

        (any other error while reading the Makefile.PL)
            A real read error, such as a disk failure, is passed on unchanged.

Warnings (processing continues):

        Warning: '$makefile' contains invalid UTF-8; reading as raw bytes: $error
            The file is not valid UTF-8.  Its raw bytes are used instead.
            Note: depending on which optional UTF-8 modules are installed
            (Unicode::UTF8, PerlIO::utf8_strict), Path::Tiny may instead
            decode the file leniently and print its own warning.  Either way
            you get a warning, not an error.
            What to do: save the Makefile.PL as UTF-8.

        Ignoring invalid version for '$module' in existing cpanfile: '$version'
            A develop entry in the existing cpanfile has a version that is
            not a version number.  The entry is kept with no version, so the
            bad value cannot change the meaning of the new cpanfile.

        No 'develop' key found in $cfg_file; using defaults
            The configuration file has no develop: section.
            What to do: add one, or delete the file.

        Skipping invalid module name in $cfg_file: '$module'
            A name in the develop: section is not a valid module name.  It is
            ignored.

        Skipping invalid version for '$module' in $cfg_file: '$version'
            A version in the develop: section is not a version number.  The
            module is kept with no minimum version.

=cut

=head2 read_makefile($path)

=head3 PURPOSE

Reads the text of a F<Makefile.PL>, exactly as L</generate(%args)> does
when it is given a C<makefile> argument.  Use it together with the
C<content> argument of C<generate()> when you need the same text for
more than one call, so that the file is read only once.

=head3 ARGUMENTS

=over 4

=item * C<$path> - as the C<makefile> argument of L</generate(%args)>:
optional, default C<'Makefile.PL'>, used as a string.

=back

=head3 RETURNS

The file's text as a character string.  If the file is not valid UTF-8
you get a warning and its raw bytes instead (see L</generate(%args)>).

=head3 SIDE EFFECTS

Reads the file.  May warn.  Does not change the caller's C<$@>, C<$!>
or C<$_>.

=head3 USAGE EXAMPLE

        my $text = App::makefilepl2cpanfile::read_makefile('Makefile.PL');
        my $deps = App::makefilepl2cpanfile::parse_prereqs($text);
        my $out  = App::makefilepl2cpanfile::generate(content => $text);

=head3 API SPECIFICATION

=head4 Input

        {
                path => {
                        type     => 'string',
                        optional => 1,
                        min      => 1,
                        default  => 'Makefile.PL',
                        position => 0,
                },
        }

=head4 Output

        {
                type => 'string',
        }

=head3 MESSAGES

The same as the file-reading messages of L</generate(%args)>:
C<Cannot read '$path'> (croak), the invalid-UTF-8 warning, and any
other read error passed on unchanged.

=cut

=head2 parse_prereqs($content)

=head3 PURPOSE

Finds every dependency listed in the text of a F<Makefile.PL> and returns
them grouped by phase and relationship.  This is the parser that
L</generate(%args)> uses; call it directly when you want the data rather
than a F<cpanfile>.  The text is never run.

=head3 ARGUMENTS

=over 4

=item * C<$content> - the text of a F<Makefile.PL>, as a character string
(see L</ENCODING>).  C<undef> or a reference is treated as text with no
dependencies.

=back

The forms that are recognised are listed in L</What it reads>.  These
rules also apply:

=over 4

=item * Only entries whose name is in quotes and is a valid module name
are used.  Several entries may be on one line; a C<#> comment at the end
of a line belongs to the last entry on that line.

=item * Commented-out code is ignored, including a whole dependency list
written on one line after a C<#>.

=item * A version must contain at least one digit and use only C<0-9>,
C<.>, C<_> and a leading C<v>; a version range joins such versions, each
after a comparison operator, with commas.  Anything else (for example
C<'.'> or C<$VERSION>) means "no minimum version".

=item * When a module appears twice in the same phase and relationship,
the first one wins.  The simple keys are read before C<prereqs> blocks.

=item * Relationship hashes inside a C<prereqs> block belong to that
block's phase; only the version 1 keys directly inside C<META_MERGE> are
mapped as described in L</What it reads>.

=item * A comment at the end of a line belongs to the last entry on the
line that is actually kept; an entry dropped for an invalid name does not
take the comment with it.

=back

=head3 RETURNS

A hash reference as described in L</DATA STRUCTURE>.  It is empty when
nothing is found.

=head3 USAGE EXAMPLE

        use App::makefilepl2cpanfile;
        use Path::Tiny;

        my $deps = App::makefilepl2cpanfile::parse_prereqs(
                path('Makefile.PL')->slurp_utf8
        );

        # Which test modules are needed, and which minimum versions?
        my $test = $deps->{test}{requires} || {};
        for my $module (sort keys %{$test}) {
                printf "%-30s %s\n", $module, $test->{$module}{version} || 'any';
        }

=head3 API SPECIFICATION

=head4 Input

        {
                content => {
                        type     => 'string',
                        optional => 1,
                        position => 0,
                },
        }

=head4 Output

        {
                type => 'hashref',
        }

=head4 Domains

        content
          valid:    any string (decoded characters; see ENCODING)
          empty:    '', undef, any reference, text with no dependency lists
                    -> {} (no warning)

        module name (the quoted key of an entry)
          valid:    ASCII letter or '_' first, then ASCII letters, digits and
                    '_', in parts joined by '::'.  Shortest: one character
                    ('A', '_').  No maximum length.
          ignored:  '' ; leading digit ('1A') ; '::' at either end ; ':::' ;
                    '-', space, ';' or the old "'" package separator ;
                    any non-ASCII character ; a bareword (unquoted) key

        version (the value of an entry, and MIN_PERL_VERSION)
          valid:    the whole value is an optional 'v' followed by ASCII
                    digits, '.' and '_', with at least one digit:
                    '1', 1.60, 'v1.2.3', '1.23_01', '5.010001'
          range:    (entries only, not MIN_PERL_VERSION) such versions, each
                    after one of >= <= == != > <, joined by commas:
                    '>= 1.2, < 2.0', '== 1.5', '< 2'.  '>= 0' means "any".
          zero:     '0', 0, '0.0', '0.000', 'v0', 'v0.0.0' -> "no minimum"
                    (nothing is written)
          invalid:  -> "no minimum": '.', '_', 'v', '1e3', '1.0-TRIAL',
                    '1 0', non-ASCII digits, $VERSION, version->parse(...),
                    spaces around the whole value, and ranges with a missing
                    comma or operator ('>= 1.2 < 2.0', '1.2, 2.0')
          limits:   no maximum length

        phase / relationship (inside prereqs blocks)
          valid:    exactly runtime, configure, build, test, develop /
                    requires, recommends, suggests, conflicts (lower case)
          ignored:  anything else, including 'Runtime', 'recommend', 'x_foo'

        comment (text after '#' on an entry's line)
          kept:     any printable Unicode: accents, 'ss'-type letters, emoji,
                    combining marks, right-to-left scripts
          removed:  control characters other than TAB (for example CR) and
                    the bidirectional control characters U+061C, U+200E,
                    U+200F, U+202A-U+202E, U+2066-U+2069, which could make the
                    generated file display differently from its real content
          empty:    a comment that is empty after this -> undef

=head3 MESSAGES

None.  Text that is not recognised is ignored without a warning.

=cut