TER1 (Statement): 100.00%
TER2 (Branch): 91.67%
TER3 (LCSAJ): 100.0% (5/5)
Approximate LCSAJ segments: 49
โ Covered โ this LCSAJ path was executed during testing.
โ Not covered โ this LCSAJ path was never executed. These are the paths to focus on.
Multiple dots on a line indicate that multiple control-flow paths begin at that line. Hovering over any dot shows:
start โ end โ jump
Uncovered paths show [NOT COVERED] in the tooltip.
1: package App::Test::Generator::Mutator; 2: 3: use strict; 4: use warnings; 5: use Carp qw(croak); 6: use Config; 7: use File::Copy qw(copy); 8: use File::Copy::Recursive qw(dircopy); 9: use File::Spec; 10: use File::Temp qw(tempdir); 11: use PPI; 12: use Readonly; 13: 14: use App::Test::Generator::Mutation::BooleanNegation; 15: use App::Test::Generator::Mutation::ConditionalInversion; 16: use App::Test::Generator::Mutation::NumericBoundary; 17: use App::Test::Generator::Mutation::ReturnUndef; 18: 19: # -------------------------------------------------- 20: # Valid mutation level values 21: # -------------------------------------------------- 22: Readonly my $LEVEL_FULL => 'full'; 23: Readonly my $LEVEL_FAST => 'fast'; 24: 25: # -------------------------------------------------- 26: # Default values for optional constructor arguments 27: # -------------------------------------------------- 28: Readonly my $DEFAULT_LIB_DIR => 'lib'; 29: Readonly my $DEFAULT_MUTATION_LEVEL => $LEVEL_FULL; 30: 31: our $VERSION = '0.41'; 32: 33: =head1 NAME 34: 35: App::Test::Generator::Mutator - Generate and apply mutation tests 36: 37: =head1 VERSION 38: 39: Version 0.41 40: 41: =head1 DESCRIPTION 42: 43: B<App::Test::Generator::Mutator> is a mutation engine that programmatically 44: alters Perl source files to evaluate the effectiveness of a project's test 45: suite. It analyses modules, generates systematic code mutations (such as 46: conditional inversions, logical operator changes, and numeric boundary 47: flips), and applies them within an isolated workspace so tests can be 48: executed safely against each modified variant. 49: 50: By tracking which mutants are killed (cause tests to fail) versus those that 51: survive (tests still pass), the module enables calculation of a mutation 52: score, providing a quantitative measure of how well the test suite detects 53: unintended behavioural changes. 54: 55: =head2 new 56: 57: Construct a new Mutator for a given source file. 58: 59: my $mutator = App::Test::Generator::Mutator->new( 60: file => 'lib/My/Module.pm', 61: lib_dir => 'lib', 62: mutation_level => 'full', 63: ); 64: 65: =head3 Arguments 66: 67: =over 4 68: 69: =item * C<file> 70: 71: Path to the Perl source file to mutate. Required. Must exist on disk. 72: 73: =item * C<lib_dir> 74: 75: Root library directory. Optional - defaults to C<lib>. 76: 77: =item * C<mutation_level> 78: 79: Controls the breadth of mutation. C<full> applies all mutations; 80: C<fast> deduplicates and removes redundant mutants first. 81: Optional - defaults to C<full>. 82: 83: =back 84: 85: =head3 Returns 86: 87: A blessed hashref. Croaks if C<file> is missing or does not exist. 88: 89: =head3 API specification 90: 91: =head4 input 92: 93: { 94: file => { type => SCALAR }, 95: lib_dir => { type => SCALAR, optional => 1 }, 96: mutation_level => { type => SCALAR, optional => 1 }, 97: } 98: 99: =head4 output 100: 101: { 102: type => OBJECT, 103: isa => 'App::Test::Generator::Mutator', 104: } 105: 106: =cut 107: 108: sub new { 109: my ($class, %args) = @_; 110: 111: # file is required and must exist on disk 112: croak 'file required' unless defined $args{file}; 113: croak "file not found: $args{file}" unless -f $args{file}; 114: 115: return bless { 116: file => $args{file}, 117: lib_dir => $args{lib_dir} || $DEFAULT_LIB_DIR, 118: mutation_level => $args{mutation_level} || $DEFAULT_MUTATION_LEVEL, 119: 120: # Instantiate all registered mutation strategies 121: mutations => [ 122: App::Test::Generator::Mutation::BooleanNegation->new(), 123: App::Test::Generator::Mutation::ReturnUndef->new(), 124: App::Test::Generator::Mutation::NumericBoundary->new(), 125: App::Test::Generator::Mutation::ConditionalInversion->new(), 126: ], 127: }, $class; 128: } 129: 130: =head2 generate_mutants 131: 132: Parse the target file and generate all mutants by running each registered 133: mutation strategy against the PPI document. 134: 135: my @mutants = $mutator->generate_mutants(); 136: 137: =head3 Arguments 138: 139: None beyond C<$self>. 140: 141: =head3 Returns 142: 143: =head3 Returns 144: 145: A list of L<App::Test::Generator::Mutant> objects. In C<fast> mode, 146: redundant and duplicate mutants are removed before returning. 147: Lines within C<## MUTANT_SKIP_BEGIN> / C<## MUTANT_SKIP_END> annotation 148: blocks are excluded from the candidate list entirely. 149: After this method returns, 150: C<$self-E<gt>{skip_lines}> contains a hashref mapping excluded 151: line numbers to 1. 152: 153: =head3 API specification 154: 155: =head4 input 156: 157: { 158: self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' }, 159: } 160: 161: =head4 output 162: 163: { 164: type => ARRAYREF, 165: elements => { type => OBJECT, isa => 'App::Test::Generator::Mutant' }, 166: } 167: 168: =cut 169: 170: sub generate_mutants { โ171 โ 182 โ 204 171: my $self = $_[0]; 172: 173: # Parse the target file into a PPI document 174: my $doc = PPI::Document->new($self->{file}) or croak "Unable to parse $self->{file}"; 175: 176: # Build set of lines excluded by ## MUTANT_SKIP_BEGIN / ## MUTANT_SKIP_END 177: my %skip_lines; 178: my $in_skip = 0; 179: my $skip_start = 0; 180: my $line_num = 0; 181: 182: for my $line (split /\n/, $doc->serialize()) { 183: $line_num++; 184: 185: # Match only lines where the annotation is the entire content â 186: # prevents false positives in comments or POD that mention the tag 187: if($line =~ /^\s*##\s*MUTANT_SKIP_BEGIN\s*$/) {Mutants (Total: 1, Killed: 1, Survived: 0)
188: croak "$self->{file}: MUTANT_SKIP_BEGIN at line $line_num with no prior MUTANT_SKIP_END" 189: if $in_skip; 190: $in_skip = 1; 191: $skip_start = $line_num; 192: } 193: $skip_lines{$line_num} = 1 if $in_skip; 194: 195: # Match only lines where the annotation is the entire content â 196: # prevents false positives in comments or POD that mention the tag 197: if($line =~ /^\s*##\s*MUTANT_SKIP_END\s*$/) {
Mutants (Total: 1, Killed: 1, Survived: 0)
198: croak "$self->{file}: MUTANT_SKIP_END at line $line_num with no matching MUTANT_SKIP_BEGIN" 199: unless $in_skip; 200: $in_skip = 0; 201: } 202: } 203: # Unclosed MUTANT_SKIP_BEGIN is fatal โ204 โ 215 โ 221 204: croak "$self->{file}: MUTANT_SKIP_BEGIN at line $skip_start has no matching MUTANT_SKIP_END" if $in_skip; 205: 206: # Store skip lines for use by the report generator 207: $self->{skip_lines} = \%skip_lines; 208: 209: my @mutants; 210: 211: # Run each registered mutation strategy against the document, 212: # excluding any candidates on skip-annotated lines. applies_to() 213: # is a cheap pre-filter -- skip the mutate() walk entirely for 214: # strategies that have nothing to match in this document. 215: for my $mutation (@{$self->{mutations}}) { 216: next unless $mutation->applies_to($doc); 217: push @mutants, grep { !$skip_lines{$_->line} } $mutation->mutate($doc); 218: }
219: 220: # In fast mode deduplicate and remove redundant mutants โ221 โ 221 โ 225 221: if($self->{mutation_level} eq $LEVEL_FAST) { 222: return @{_dedup_mutants(\@mutants)};Mutants (Total: 1, Killed: 0, Survived: 1)
- COND_INV_218_2: Invert condition if to unless
MEDIUM: Add tests asserting both true and false outcomesMutants (Total: 2, Killed: 2, Survived: 0)
223: } 224: 225: return @mutants; 226: } 227: 228: =head2 prepare_workspace 229: 230: Prepare an isolated temporary workspace for a single mutation test run. 231: 232: The entire C<lib_dir> tree is copied into the workspace so that all module 233: dependencies resolve correctly when the test suite runs against the mutant. 234: Only after this copy is complete is the single target file overwritten by 235: C<apply_mutant>. 236: 237: my $workspace = $mutator->prepare_workspace(); 238: $mutator->apply_mutant($mutant); 239: local $ENV{PERL5LIB} = "$workspace/lib"; 240: my $survived = (system('prove', 't') == 0); 241: 242: =head3 Arguments 243: 244: None beyond C<$self>. 245: 246: =head3 Returns 247: 248: A string containing the absolute path to the temporary directory created. 249: The directory is automatically removed when the object goes out of scope 250: via L<File::Temp>'s C<CLEANUP =E<gt> 1> behaviour. 251: 252: =head3 Side effects 253: 254: Creates a temporary directory. Recursively copies C<lib_dir> into it. 255: Sets C<< $self->{workspace} >> and C<< $self->{relative} >>. 256: 257: =head3 Notes 258: 259: Call C<prepare_workspace> once per file, then C<apply_mutant> once per 260: mutant within that file. Do not store the returned path beyond the 261: lifetime of the enclosing scope. 262: 263: =head3 API specification 264: 265: =head4 input 266: 267: { 268: self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' }, 269: } 270: 271: =head4 output 272: 273: { 274: type => SCALAR, 275: } 276: 277: =cut 278: 279: sub prepare_workspace { 280: my $self = $_[0]; 281: 282: # Create a self-cleaning temporary directory 283: my $tmp = tempdir(CLEANUP => 1); 284: 285: # Normalise lib_dir to its final component so workspace paths 286: # are relative regardless of whether an absolute path was passed in 287: my $lib_basename = (File::Spec->splitdir($self->{lib_dir}))[-1]; 288: 289: # Derive the file's path relative to lib_dir for use by apply_mutant 290: my $relative = $self->{file}; 291: $relative =~ s/^\Q$self->{lib_dir}\E\/?//; 292: 293: # Copy the entire lib tree so all dependencies resolve in the workspace 294: dircopy($self->{lib_dir}, File::Spec->catfile($tmp, $lib_basename)) or croak "dircopy failed: $!"; 295: 296: $self->{workspace} = $tmp; 297: $self->{relative} = $relative;
Mutants (Total: 2, Killed: 2, Survived: 0)
298: $self->{lib_dir} = $lib_basename; # normalise for apply_mutant 299: 300: return $tmp; 301: } 302: 303: =head2 apply_mutant 304: 305: Apply a single mutant's transform to the target file in the workspace. 306: 307: $mutator->apply_mutant($mutant); 308: 309: =head3 Arguments 310: 311: =over 4 312: 313: =item * C<$mutant> 314: 315: An L<App::Test::Generator::Mutant> object whose C<transform> closure 316: will be applied to the workspace copy of the target file. 317: 318: =back 319: 320: =head3 Returns 321: 322: Nothing. Modifies the workspace copy of the target file in place. 323: 324: =head3 Side effects 325: 326: Overwrites the target file in the workspace with the mutated version. 327: 328: =head3 API specification 329: 330: =head4 input 331: 332: { 333: self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' }, 334: mutant => { type => OBJECT, isa => 'App::Test::Generator::Mutant' }, 335: } 336: 337: =head4 output 338: 339: { type => UNDEF } 340: 341: =cut 342: 343: sub apply_mutant { 344: my ($self, $mutant) = @_; 345: 346: # Workspace must be prepared before applying any mutant 347: my $workspace = $self->{workspace} 348: or croak 'Workspace not prepared â call prepare_workspace first'; 349: 350: my $relative = $self->{relative} 351: or croak 'Relative path not set â call prepare_workspace first'; 352: 353: # Construct the full path to the file in the workspace 354: my $target = File::Spec->catfile( 355: $workspace, 356: $self->{lib_dir}, 357: $relative, 358: ); 359: 360: # Parse the workspace copy and apply the mutation transform 361: my $doc = PPI::Document->new($target) or croak "Failed to parse $target"; 362: 363: $mutant->transform->($doc); 364: 365: $doc->save($target); 366: } 367: 368: =head2 run_tests 369: 370: Run the test suite against the current workspace and return whether all 371: tests passed. 372: 373: my $survived = $mutator->run_tests(); 374: 375: =head3 Arguments 376: 377: None beyond C<$self>. 378: 379: =head3 Returns 380: 381: 1 if all tests passed (mutant survived), 0 if any test failed (mutant 382: killed). 383: 384: =head3 Side effects 385: 386: Executes an external process running the test suite. 387: 388: =head3 Notes 389: 390: Uses C<prove> found on PATH. Sets C<PERL5LIB> to include the workspace 391: lib directory before running. 392: 393: =head3 API specification 394: 395: =head4 input 396: 397: { 398: self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' }, 399: } 400: 401: =head4 output 402: 403: { type => SCALAR } 404: 405: =cut 406: 407: sub run_tests { 408: my $self = $_[0]; 409: 410: # Locate prove on PATH â fall back to bare 'prove' and let shell find it 411: my $prove = File::Spec->catfile($Config{bin}, 'prove');
412: $prove = 'prove' unless -x $prove; 413: 414: return system($prove, '-l', 't') == 0; 415: } 416: 417: # -------------------------------------------------- 418: # _dedup_mutants 419: # 420: # Purpose: Remove duplicate and redundant mutants 421: # from a list, used in fast mutation mode 422: # to reduce the number of mutants to run. 423: # 424: # Entry: $mutants - arrayref of Mutant objects. 425: # 426: # Exit: Returns an arrayref of deduplicated 427: # Mutant objects. 428: # 429: # Side effects: None. 430: # 431: # Notes: Deduplication key uses line, original, 432: # and description rather than the transform 433: # coderef, which is not stable as a string. 434: # -------------------------------------------------- 435: sub _dedup_mutants { โ436 โ 440 โ 453 436: my ($mutants) = @_; 437: my @rc; 438: my %seen; 439: 440: for my $m (@{$mutants}) { 441: # Build a stable key from metadata â not from the coderef 442: my $key = join '|', 443: $m->line // '', 444: $m->original // '', 445: $m->description // ''; 446: 447: next if $seen{$key}++; 448: next if _is_redundant_mutation($m); 449: 450: push @rc, $m; 451: } 452: 453: return \@rc; 454: } 455: 456: # -------------------------------------------------- 457: # _is_redundant_mutation 458: # 459: # Return true if a mutant is considered 460: # redundant and should be skipped in fast 461: # mutation mode. 462: # 463: # Entry: $m - a Mutant object. 464: # 465: # Exit: Returns 1 if redundant, 0 otherwise. 466: # 467: # Notes: Checks for arithmetic no-ops, double 468: # negation inside conditionals, boolean 469: # literal flips, mutations inside comments, 470: # and equivalent numeric comparisons. 471: # Does not compare transform coderefs â 472: # they are not meaningful as strings. 473: # -------------------------------------------------- 474: sub _is_redundant_mutation { โ475 โ 485 โ 490 475: my ($m) = @_; 476: 477: my $orig = $m->original // '';Mutants (Total: 1, Killed: 0, Survived: 1)
- NUM_BOUNDARY_411_35_!=: Numeric boundary flip == to !=
HIGH: Likely missing edge-case test (boundary value)๐งช Suggested Test# Boundary test suggestion is( func(VALUE_AT_BOUNDARY), EXPECTED, 'Test boundary behaviour' );Mutants (Total: 2, Killed: 2, Survived: 0)
478:
Mutants (Total: 2, Killed: 2, Survived: 0)
479: # Arithmetic no-ops add nothing to mutation coverage 480: return 1 if $orig =~ /\+\s*0$/; 481: return 1 if $orig =~ /-\s*0$/; 482:
Mutants (Total: 1, Killed: 1, Survived: 0)
483: # Double negation inside conditionals forces boolean context
Mutants (Total: 2, Killed: 2, Survived: 0)
484: # in Perl and is not a meaningful mutation 485: if($m->context && $m->context eq 'conditional') { 486: return 1 if $orig =~ /^\!\!/; 487: }
Mutants (Total: 2, Killed: 2, Survived: 0)
488: 489: # Boolean literal flip on a standalone 1 or 0 is trivial 490: return 1 if $orig =~ /^\s*(?:1|0)\s*$/;
Mutants (Total: 2, Killed: 2, Survived: 0)
491: 492: # Mutations inside comments are unreachable code
Mutants (Total: 2, Killed: 2, Survived: 0)
493: return 1 if $m->line_content && $m->line_content =~ /^\s*#/; 494: 495: return 0; 496: } 497: 498: =head1 SEE ALSO 499: 500: =over 4 501: 502: =item C<bin/test-generator-mutate> 503: 504: =item L<Devel::Mutator> 505: 506: =back 507: 508: =head1 AUTHOR 509: 510: Nigel Horne, C<< <njh at nigelhorne.com> >> 511: 512: =head1 LICENCE AND COPYRIGHT 513: 514: Copyright 2026 Nigel Horne. 515: 516: Usage is subject to the terms of GPL2. 517: If you use it, 518: please let me know. 519: 520: =cut 521: 522: 1;