File Coverage

=head1 NAME

App::Test::Generator - Generate fuzz and corpus-driven test harnesses

=head1 SYNOPSIS

From the command line:

  fuzz-harness-generator t/conf/add.conf > t/add_fuzz.t

From Perl:

  use App::Test::Generator qw(generate);

  # Generate to STDOUT
  App::Test::Generator::generate("t/conf/add.conf");

  # Generate directly to a file
  App::Test::Generator::generate('t/conf/add.conf', 't/add_fuzz.t');

=head1 OVERVIEW

This module takes a formal input/output specification for a routine or
method and automatically generates test cases. In effect, it allows you
to easily add comprehensive black-box tests in addition to the more
common white-box tests that are typically written for CPAN modules and other
subroutines.

The generated tests combine:

=over 4

=item * Random fuzzing based on input types

=item * Deterministic edge cases for min/max constraints

=item * Static corpus tests defined in Perl or YAML

=back

This approach strengthens your test suite by probing both expected and
unexpected inputs, helping you to catch boundary errors, invalid data
handling, and regressions without manually writing every case.

=head1 DESCRIPTION

This module implements the logic behind L<fuzz-harness-generator>.
It parses configuration files (fuzz and/or corpus YAML), and
produces a ready-to-run F<.t> test script using L<Test::Most>.

It reads configuration files (Perl C<.conf> with C<our> variables,
and optional YAML corpus files), and generates a L<Test::Most>-based
fuzzing harness combining:

=over 4

=item * Randomized fuzzing of inputs (with edge cases)

=item * Optional static corpus tests from Perl C<%cases> or YAML file (C<yaml_cases> key)

=item * Functional or OO mode (via C<$new>)

=item * Reproducible runs via C<$seed> and configurable iterations via C<$iterations>

=back

=head2 EDGE CASE GENERATION

In addition to purely random fuzz cases, the harness generates
deterministic edge cases for parameters that declare C<min>, C<max> or C<len> in their schema definitions.

For each constraint, three edge cases are added:

=over 4

=item * Just inside the allowable range

This case should succeed, since it lies strictly within the bounds.

=item * Exactly on the boundary

This case should succeed, since it meets the constraint exactly.

=item * Just outside the boundary

This case is annotated with C<_STATUS = 'DIES'> in the corpus and
should cause the harness to fail validation or croak.

=back

Supported constraint types:

=over 4

=item * C<number>, C<integer>

Uses numeric values one below, equal to, and one above the boundary.

=item * C<string>

Uses strings of lengths one below, equal to, and one above the boundary.

=item * C<arrayref>

Uses references to arrays of with the number of elements one below, equal to, and one above the boundary.

=item * C<hashref>

Uses hashes with key counts one below, equal to, and one above the
boundary (C<min> = minimum number of keys, C<max> = maximum number
of keys).

=item * C<memberof> - arrayref of allowed values for a parameter:

    our %input = (
        status => { type => 'string', memberof => [ 'ok', 'error', 'pending' ] },
        level => { type => 'integer', memberof => [ 1, 2, 3 ] },
    );

The generator will automatically create test cases for each allowed value (inside the member list),
and at least one value outside the list (which should die, C<_STATUS = 'DIES'>).
This works for strings, integers, and numbers.

=item * C<boolean> - automatic boundary tests for boolean fields

    our %input = (
        flag => { type => 'boolean' },
    );

The generator will automatically create test cases for 0 and 1; true and false; off and on, and values that should trigger C<_STATUS = 'DIES'>.

=back

These edge cases are inserted automatically, in addition to the random
fuzzing inputs, so each run will reliably probe boundary conditions
without relying solely on randomness.

=head1 CONFIGURATION

The configuration file is either a file that can be read by L<Config::Abstraction> or a B<trusted input> Perl file that should set variables with C<our>.

The documentation here covers the old trusted input style input, but that will go away so you are recommended to use
Config::Abstraction files.
Example: the generator expects your config to use C<our %input>, C<our $function>, etc.

Recognized items:

=over 4

=item * C<%input> - input params with keys => type/optional specs:

When using named parameters
        our %input = (
                name => { type => 'string', optional => 0 },
                age => { type => 'integer', optional => 1 },
        );

Supported basic types used by the fuzzer: C<string>, C<integer>, C<number>, C<boolean>, C<arrayref>, C<hashref>.
(You can add more types; they will default to C<undef> unless extended.)

For routines with one unnamed parameter

  our %input = (
     type => 'string'
  );

Currently, routines with more than one unnamed parameter are not supported.

=item * C<%output> - output param types for Return::Set checking:

        our %output = (
                type => 'string'
        );

If the output hash contains the key _STATUS, and if that key is set to DIES,
the routine should die with the given arguments; otherwise, it should live.
If it's set to WARNS,
the routine should warn with the given arguments

=item * C<$module> - module name (optional).

If omitted, the generator will guess from the config filename:
C<My-Widget.conf> -> C<My::Widget>.

=item * C<$function> - function/method to test (defaults to C<run>).

=item * C<$new> - optional hashref of args to pass to the module's constructor (object mode):

        our $new = { api_key => 'ABC123', verbose => 1 };

To ensure new is called with no arguments, you still need to define new, thus:

  our $new = '';

=item * C<%cases> - optional Perl static corpus, when the output is a simple string (expected => [ args... ]):

Maps the expected output string to the input and _STATUS

  our %cases = (
    'ok'   => {
        input => 'ping',
        status => 'OK',
    'error' =>
        input => '',
        status => 'DIES'
  );

=item * C<$yaml_cases> - optional path to a YAML file with the same shape as C<%cases>.

=item * C<$seed> - optional integer. When provided, the generated C<t/fuzz.t> will call C<srand($seed)> so fuzz runs are reproducible.

=item * C<$iterations> - optional integer controlling how many fuzz iterations to perform (default 50).

=item * C<%edge_cases> - optional hash mapping of extra values to inject:

        # Two named parameters
        our %edge_cases = (
                name => [ '', 'a' x 1024, \"\x{263A}" ],
                age  => [ -1, 0, 99999999 ],
        );

        # Takes a string input
        our %edge_cases (
                'foo', 'bar'
        );

(Values can be strings or numbers; strings will be properly quoted.)
Note that this only works with routines that take named parameters.

=item * C<%type_edge_cases> - optional hash mapping types to arrayrefs of extra values to try for any field of that type:

        our %type_edge_cases = (
                string  => [ '', ' ', "\t", "\n", "\0", 'long' x 1024, chr(0x1F600) ],
                number  => [ 0, 1.0, -1.0, 1e308, -1e308, 1e-308, -1e-308, 'NaN', 'Infinity' ],
                integer => [ 0, 1, -1, 2**31-1, -(2**31), 2**63-1, -(2**63) ],
        );

=item * C<%config> - optional hash of configuration.

The current supported variables are

=over 4

=item * C<test_nuls>, inject NUL bytes into strings (default: 1)

=item * C<test_undef>, test with undefined value (default: 1)

=item * C<dedup>, fuzzing can create duplicate tests, go some way to remove duplicates (default: 1)

=back

=back

=head1 EXAMPLES

=head2 Math::Simple::add()

Functional fuzz + Perl corpus + seed:

  our $module = 'Math::Simple';
  our $function = 'add';
  our %input = ( a => { type => 'integer' }, b => { type => 'integer' } );
  our %output = ( type => 'integer' );
  our %cases = (
    '3'     => [1, 2],
    '0'     => [0, 0],
    '-1'    => [-2, 1],
    '_STATUS:DIES'  => [ 'a', 'b' ],     # non-numeric args should die
    '_STATUS:WARNS' => [ undef, undef ], # undef args should warn
  );
  our $seed = 12345;
  our $iterations = 100;

=head2 Adding YAML file to generate tests

OO fuzz + YAML corpus + edge cases:

        our %input = ( query => { type => 'string' } );
        our %output = ( type => 'string' );
        our $function = 'search';
        our $new = { api_key => 'ABC123' };
        our $yaml_cases = 't/corpus.yml';
        our %edge_cases = ( query => [ '', '     ', '<script>' ] );
        our %type_edge_cases = ( string => [ \"\\0", "\x{FFFD}" ] );
        our $seed = 999;

=head3 YAML Corpus Example (t/corpus.yml)

A YAML mapping of expected -> args array:

        "success":
          - "Alice"
          - 30
        "failure":
          - "Bob"

=head2 Example with arrayref + hashref

  our %input = (
    tags   => { type => 'arrayref', optional => 1 },
    config => { type => 'hashref' },
  );
  our %output = ( type => 'hashref' );

=head2 Example with memberof

  our %input = (
      status => { type => 'string', memberof => [ 'ok', 'error', 'pending' ] },
  );
  our %output = ( type => 'string' );
  our %config = ( test_nuls => 0, test_undef => 1 );

This will generate fuzz cases for 'ok', 'error', 'pending', and one invalid string that should die.

=head2 New format input

Testing L<HTML::Genealogy::Map>:

  ---

  module: HTML::Genealogy::Map
  function: onload_render

  input:
    gedcom:
      type: object
      can: individuals
    geocoder:
      type: object
      can: geocode
    debug:
      type: boolean
      optional: true
    google_key:
      type: string
      optional: true
      min: 39
      max: 39
      matches: "^AIza[0-9A-Za-z_-]{35}$"

  config:
    test_undef: 0

=head1 OUTPUT

By default, writes C<t/fuzz.t>.
The generated test:

=over 4

=item * Seeds RNG (if configured) for reproducible fuzz runs

=item * Uses edge cases (per-field and per-type) with configurable probability

=item * Runs C<$iterations> fuzz cases plus appended edge-case runs

=item * Validates inputs with Params::Get / Params::Validate::Strict

=item * Validates outputs with L<Return::Set>

=item * Runs static C<is(... )> corpus tests from Perl and/or YAML corpus

=back

=head1 NOTES

=over 4

=item * The conf file must use C<our> declarations so variables are visible to the generator via C<require>.

=back

=cut

=head1 SEE ALSO

=over 4

=item * Test coverage report: L<https://nigelhorne.github.io/App-Test-Generator/coverage/>

=item * L<Test::Most>, L<Params::Get>, L<Params::Validate::Strict>, L<Return::Set>, L<YAML::XS>

=back

=head1 AUTHOR

Nigel Horne, C<< <njh at nigelhorne.com> >>

Portions of this module's design and documentation were created with the
assistance of L<ChatGPT|https://openai.com/> (GPT-5), with final curation
and authorship by Nigel Horne.

=cut