File Coverage

File:blib/lib/Database/Abstraction.pm
Coverage:80.2%

linestmtbrancondsubtimecode
1package Database::Abstraction;
2
3# Author Nigel Horne: njh@nigelhorne.com
4# Copyright (C) 2015-2026, Nigel Horne
5
6# Usage is subject to licence terms.
7# The licence terms of this software are as follows:
8# Personal single user, single computer use: GPL2
9# All other users (for example, Commercial, Charity, Educational, Government)
10#       must apply in writing for a licence for use from Nigel Horne at the
11#       above e-mail.
12
13# TODO: Switch "entry" to off by default, and enable by passing 'entry'
14#       though that wouldn't be so nice for AUTOLOAD
15# TODO: support a directory hierarchy of databases
16# TODO: consider returning an object or array of objects, rather than hashes
17# TODO: Add redis database - could be of use for Geo::Coder::Free
18#       use select() to select a database - use the table arg
19#       new(database => 'redis://servername');
20# TODO: Add a "key" property, defaulting to "entry", which would be the name of the key
21# TODO: The maximum number to return should be tuneable (as a LIMIT)
22# TODO: Add full CRUD support
23# TODO: It would be better for the default sep_char to be ',' rather than '!'
24# TODO: Other databases e.g., Redis, noSQL, remote databases such as MySQL, PostgreSQL
25# TODO: The no_entry/entry terminology is confusing.  Replace with no_id/id_column
26# TODO: Add support for DBM::Deep
27# TODO: Log queries and the time that they took to execute per database
28
29
28
28
28
2800543
24
603
use warnings;
30
28
28
28
51
21
275
use strict;
31
28
28
28
4348
146257
59
use autodie qw(:all);
32
33
28
28
28
193957
12522
58
use boolean;
34
28
28
28
881
23
538
use Carp;
35
28
28
28
5719
524381
85
use Class::Abstract;
36
28
28
28
9229
41013
631
use Data::Reuse;
37
28
28
28
20433
212814
920
use DBI;
38
28
28
28
89
22
3050
use Fcntl;      # For O_RDONLY
39
28
28
28
63
22
720
use Cwd;
40
28
28
28
53
23
323
use File::Spec;
41
28
28
28
6742
120709
970
use File::Temp;
42
28
28
28
65
23
830
use List::Util qw(all);
43
28
28
28
8358
950598
665
use Log::Abstraction 0.26;
44
28
28
28
7902
142659
555
use Object::Configure 0.16;
45
28
28
28
106
151
503
use Params::Get 0.13;
46
28
28
28
56
21
515
use Return::Set qw(set_return);
47
28
28
28
44
25
557
use Scalar::Util;
48
49our %defaults;
50
28
28
28
47
24
153893
use constant    DEFAULT_MAX_SLURP_SIZE => 16 * 1024; # CSV files <= than this size are read into memory
51
52 - 60
=head1 NAME

Database::Abstraction - Read-only Database Abstraction Layer (ORM)

=head1 VERSION

Version 0.36

=cut
61
62our $VERSION = '0.36';
63
64 - 338
=head1 DESCRIPTION

C<Database::Abstraction> is a read-only ORM for Perl that gives a uniform
interface over CSV, PSV, XML, SQLite, and BerkeleyDB files - without writing
any SQL.

Key features:

=over 4

=item *

B<No SQL required.>  Use plain Perl method calls for simple lookups and
scans; switch storage formats without changing application code.

=item *

B<Rich query criteria.>  Pass plain values, SQL wildcards, C<undef> (IS NULL),
comparison operators (C<< > >> C<< < >> C<< >= >> C<< <= >> C<!=>), pattern
operators (C<-like>, C<-not_like>), set operators (C<-in>, C<-not_in>,
C<-between>), and logical groupings (C<-or>, C<-and>).

=item *

B<Automatic joins.>  Add a C<join> parameter to any select method to
combine tables with INNER, LEFT, RIGHT, FULL, or CROSS joins.

=item *

B<Chained query builder.>  The C<query()> method returns a
L<Database::Abstraction::Query> object for fluent, composable queries:
C<< $db->query->where(...)->order_by(...)->limit(...)->all() >>.

=item *

B<Schema introspection.>  C<columns()> lists column names; C<schema()>
returns full type/nullability metadata, using native driver introspection
(C<PRAGMA table_info> for SQLite, C<column_info> for others).

=item *

B<DSN portability.>  Pass a C<dsn> (plus optional C<username>/C<password>)
to connect to any DBI-supported database (SQLite, PostgreSQL, MySQL, ...)
instead of pointing at a local file.

=item *

B<Performance.>  Small files are slurped into a RAM hash for sub-millisecond
lookups.  All DBI statement handles are cached with C<prepare_cached()>.
A CHI-compatible cache layer is also supported.

=back

=head1 SYNOPSIS

    # 1. Create a thin subclass for your table (e.g. Database/Foo.pm)
    package Database::Foo;
    use parent 'Database::Abstraction';

    # 2. Open the database - file is auto-detected from the class name
    #    (looks for foo.sql / foo.psv / foo.csv / foo.xml / foo.db)
    my $db = Database::Foo->new(directory => '/path/to/data');

    # 3. Simple lookups -----------------------------------------------

    # Fetch one row
    my $row = $db->fetchrow_hashref(entry => 'key1');

    # Fetch all rows matching a criterion
    my $rows = $db->selectall_arrayref(status => 'active');

    # Column shortcut via AUTOLOAD
    my $name = $db->name(entry => 'key1');

    # 4. Rich criteria ------------------------------------------------

    # Comparison operators
    my $high = $db->selectall_arrayref(score => { '>' => 90 });

    # Set membership
    my $selected = $db->selectall_arrayref(
        name => { -in => ['Alice', 'Bob'] }
    );

    # Range
    my $mid = $db->selectall_arrayref(
        score => { -between => [60, 80] }
    );

    # OR grouping
    my $either = $db->selectall_arrayref(
        -or => [
            { status => 'active'    },
            { score  => { '>' => 95 } },
        ]
    );

    # 5. Joins --------------------------------------------------------

    my $joined = $db->selectall_arrayref(
        join => { table => 'dept', on => 'foo.dept_id = dept.id', type => 'LEFT' }
    );

    # 6. Chained query builder ----------------------------------------

    my $results = $db->query
        ->where(status => 'active')
        ->where(score  => { '>=' => 80 })
        ->order_by('score DESC')
        ->limit(10)
        ->all();

    my $first = $db->query->where(name => 'Alice')->first();
    my $count = $db->query->where(status => 'active')->count();

    # 7. Connect via DSN (PostgreSQL, MySQL, SQLite, ...) ---------------

    my $db2 = Database::Foo->new(
        dsn      => 'dbi:Pg:dbname=mydb;host=db.example.com',
        username => 'myuser',
        password => 's3cret',
    );

    # 8. Schema introspection -----------------------------------------

    my $cols   = $db->columns();  # ['entry', 'name', 'score', ...]
    my $schema = $db->schema();   # { name => { type=>'TEXT', nullable=>1, ... }, ... }

=head1 QUICK START EXAMPLE

If F</var/dat/foo.csv> contains:

    "customer_id","name"
    "plugh","John"
    "xyzzy","Jane"

Create a driver in F<.../Database/foo.pm>:

    package Database::foo;
    use parent 'Database::Abstraction';

    # Regular CSV: no entry column, comma-separated
    sub new {
        my ($class, %args) = @_;
        return $class->SUPER::new(no_entry => 1, sep_char => ',', %args);
    }

Then query it:

    my $foo = Database::foo->new(directory => '/var/dat');

    # Prints "John"
    print 'Customer: ', $foo->name(customer_id => 'plugh'), "\n";

    # Returns { customer_id => 'xyzzy', name => 'Jane' }
    my $row = $foo->fetchrow_hashref(customer_id => 'xyzzy');

=head1 FILE FORMATS

The module probes the C<directory> for files in this priority order:

=over 4

=item 1. C<SQLite>

File ending C<.sql>

=item 2. C<PSV>

Pipe-separated file, ending C<.psv>

=item 3. C<CSV>

Comma (or custom) separated file, ending C<.csv> or C<.db>; can be
gzipped.  B<Note:> the default separator is C<!> not C<,> for historical
reasons - pass C<< sep_char => ',' >> for standard CSVs.

=item 4. C<XML>

File ending C<.xml>

=item 5. C<BerkeleyDB>

Binary key-value file ending C<.db>

=back

Pass C<dsn> to bypass file detection entirely and connect via any DBI driver.

=head1 QUERY CRITERIA

All select methods (C<selectall_arrayref>, C<selectall_array>,
C<fetchrow_hashref>, C<count>) accept the same criteria syntax.

=head2 Plain value

    status => 'active'          # status = 'active'
    name   => undef             # name IS NULL

Values containing C<%> or C<_> are matched with C<LIKE>:

    name => 'A%'                # name LIKE 'A%'

=head2 Comparison operator hashref

    score => { '>'  => 90  }   # score > 90
    score => { '<'  => 50  }   # score < 50
    score => { '>=' => 80  }   # score >= 80
    score => { '<=' => 100 }   # score <= 100
    score => { '!=' => 0   }   # score != 0

Multiple operators on one column are ANDed:

    score => { '>' => 60, '<' => 90 }   # 60 < score < 90

=head2 Pattern matching

    name => { -like     => 'A%'  }   # name LIKE 'A%'
    name => { -not_like => 'Z%'  }   # name NOT LIKE 'Z%'

=head2 Set membership

    name => { -in     => ['Alice', 'Bob'] }   # name IN (...)
    name => { -not_in => ['Alice', 'Bob'] }   # name NOT IN (...)

=head2 Range

    score => { -between => [60, 90] }   # score BETWEEN 60 AND 90

=head2 Logical groupings

C<-or> and C<-and> take an arrayref of condition hashrefs:

    -or => [
        { status => 'active'        },
        { score  => { '>' => 95 }   },
    ]

    -and => [
        { status => 'active'        },
        { score  => { '>=' => 80 }  },
    ]

=head2 Joins

Any select method accepts a C<join> key with a hashref (or arrayref of
hashrefs) describing the join:

    join => {
        table => 'dept',
        on    => 'employees.dept_id = dept.id',
        type  => 'LEFT',    # INNER (default) | LEFT | RIGHT | FULL | CROSS
    }

    # Multiple joins
    join => [
        { table => 'dept',    on => 'e.dept_id   = dept.id'   },
        { table => 'country', on => 'e.country_id = country.id' },
    ]

=head1 SUBROUTINES/METHODS

=head2 init

Set class-level defaults shared by all instances.

    Database::Abstraction::init(directory => '../data');

Accepts the same parameters as L</new>.  Returns a reference to the
current defaults hash, so you can read them back:

    my $defaults = Database::Abstraction::init();
    print $defaults->{'directory'}, "\n";

=cut
339
340# Subroutine to initialize with args
341sub init
342{
343
111
764454
        if(my $params = Params::Get::get_params(undef, @_)) {
344
105
1401
                if(($params->{'expires_in'} && !$params->{'cache_duration'})) {
345                        # Compatibility with CHI
346
4
5
                        $params->{'cache_duration'} = $params->{'expires_in'};
347                }
348
349
105
105
747
248
                %defaults = (%defaults, %{$params});
350
105
382
                $defaults{'cache_duration'} ||= '1 hour';
351        }
352
353
111
6974
        return \%defaults
354}
355
356 - 366
=head2 import

The module can be initialised by the C<use> directive.

    use Database::Abstraction 'directory' => '/etc/data';

or

    use Database::Abstraction { 'directory' => '/etc/data' };

=cut
367
368sub import
369{
370
88
650040
        my $pkg = shift;
371
372
88
200
        if((scalar(@_) % 2) == 0) {
373
86
101
                my %h = @_;
374
86
181
                init(Object::Configure::configure($pkg, \%h));
375        } elsif((scalar(@_) == 1) && (ref($_[0]) eq 'HASH')) {
376
2
6
                init(Object::Configure::configure($pkg, $_[0]));
377        } elsif(scalar(@_) > 0) {    # >= 3 would also work here
378
0
0
                init(\@_);
379        }
380}
381
382 - 510
=head2 new

Create an object pointing to a read-only database.

Accepts arguments as a hash, a hashref, or - as a shortcut - a single bare
string which is taken to be C<directory>.

=head3 Connection parameters

=over 4

=item * C<directory>

Directory containing the data files.  The module probes this directory for
files named after the subclass (see L</FILE FORMATS>).  Required unless
C<dsn> is given.

=item * C<dsn>

A DBI data-source string (e.g. C<dbi:SQLite:dbname=/path/to/db> or
C<dbi:Pg:dbname=mydb;host=db.example.com>).  When present, file detection
is skipped entirely and the DSN is used directly.  The SQL dialect is
inferred from the DSN prefix (C<sqlite>, C<postgres>, C<mysql>).

=item * C<username>

Database username.  Used only with C<dsn>; ignored for file-based backends.

=item * C<password>

Database password.  Used only with C<dsn>; ignored for file-based backends.

=item * C<dbname>

Override the filename stem searched in C<directory> (default: the table
name derived from the class name).

=item * C<filename>

Override the full filename (relative to C<directory>).  Takes precedence
over C<dbname>.

=back

=head3 Behaviour parameters

=over 4

=item * C<no_entry>

Set to C<1> when the table has no key column (standard CSVs, for example).
Default is C<0> (keyed on C<entry>).

=item * C<id>

Name of the key column.  Default is C<entry>.

=item * C<sep_char>

Field separator for CSV/PSV files.
Default is C<!> - pass C<< sep_char => ',' >>
for standard comma-separated files.

=item * C<max_slurp_size>

Files smaller than this (in bytes) are loaded entirely into memory for fast
lookups.  Default is 16 KB.  Set to C<0> to force SQL mode for all sizes.

=item * C<no_fixate>

Set to C<1> to return mutable arrays.  Default is C<0> (arrays are made
read-only via L<Data::Reuse>).

=item * C<auto_load>

Set to C<0> to disable the AUTOLOAD column shortcut.  Default is C<1>
(enabled).

=back

=head3 Caching and logging

=over 4

=item * C<cache>

A L<CHI>-compatible cache object.  When set, query results are stored and
retrieved from the cache.

=item * C<cache_duration> / C<expires_in>

TTL for cached results.  Default is C<'1 hour'>.  C<expires_in> is a
synonym for compatibility with L<CHI>.

=item * C<logger>

An object that understands C<warn()> and C<trace()> (e.g.
L<Log::Log4perl>, L<Log::Any>), a code reference, or a filename.

=item * C<config_file>

Path to a YAML, XML, or INI configuration file whose keys are merged into
the constructor arguments.  Loaded via L<Object::Configure>.

=back

=head3 Notes

=over 4

=item *

If no arguments are set, class-level defaults set via C<init()> or C<use>
are used.

=item *

Slurp mode assumes the key column (C<entry>) is unique.  If it is not,
searches will be incomplete - disable slurp mode by setting
C<< max_slurp_size => 0 >>.

=item *

Passing an existing object as C<$class> clones it, merging any new
arguments.

=back

=cut
511
512sub new {
513
210
1310896
        my $class = shift;
514
210
197
        my %args;
515
516
210
471
        Class::Abstract::check_abstract($class);        # enforces abstract contract
517
518        # Handle hash or hashref arguments
519
210
1790
        if((scalar(@_) == 1) && !ref($_[0])) {
520
56
202
                $args{'directory'} = $_[0];
521        } elsif(my $params = Params::Get::get_params(undef, @_)) {
522
144
144
1986
261
                %args = %{$params};
523        }
524
525
210
1271
        if(!defined($class)) {
526
0
0
                if((scalar keys %args) > 0) {
527                        # Using Database::Abstraction->new(), not Database::Abstraction::new()
528
0
0
                        carp(__PACKAGE__, ' use ->new() not ::new() to instantiate');
529
0
0
                        return;
530                }
531                # FIXME: this only works when no arguments are given
532
0
0
                $class = __PACKAGE__;
533        } elsif($class eq __PACKAGE__) {
534
6
57
                croak("$class: abstract class");
535        } elsif(Scalar::Util::blessed($class)) {
536                # If $class is an object, clone it with new arguments
537
6
6
6
24
                return bless { %{$class}, %args }, ref($class);
538        }
539
540        # Load the configuration from a config file, if provided
541
198
198
187
375
        %args = %{Object::Configure::configure($class, \%args)};
542
543        # Normalise logger: wrap code-refs, filenames, and strings in Log::Abstraction
544        # so that the rest of the code can always call ->$level(...) uniformly.
545
198
384504
        if(defined $args{'logger'} && !Scalar::Util::blessed($args{'logger'})) {
546
0
0
                $args{'logger'} = Log::Abstraction->new($args{'logger'});
547        }
548
549
198
527
        unless($args{'dsn'} || $defaults{'dsn'}) {
550
182
385
                croak("$class: where are the files?") unless($args{'directory'} || $defaults{'directory'});
551
552
177
1258
                croak("$class: ", $args{'directory'} || $defaults{'directory'}, ' is not a directory') unless(-d ($args{'directory'} || $defaults{'directory'}));
553        }
554
555        # Validate the primary-key column name to prevent SQL injection via ORDER BY / WHERE
556
187
276
        for my $src (\%defaults, \%args) {
557
374
1017
                if(defined $src->{'id'}) {
558                        croak("$class: unsafe id column name '$src->{id}'")
559
14
60
                                unless $src->{'id'} =~ /^[a-zA-Z_][a-zA-Z0-9_]*$/;
560                }
561        }
562
563        # Defaults are set first so that %args keys override them
564
187
906
        return bless {
565                no_entry => 0,
566                no_fixate => 0,
567                id => 'entry',
568                cache_duration => '1 hour',
569                max_slurp_size => DEFAULT_MAX_SLURP_SIZE,
570                %defaults,
571                %args,
572        }, $class;
573}
574
575 - 579
=head2  set_logger

Sets the class, code reference, or file that will be used for logging.

=cut
580
581sub set_logger
582{
583
16
4040
        my $self = shift;
584
16
29
        my $params = Params::Get::get_params('logger', @_);
585
586
13
192
        if(my $logger = $params->{'logger'}) {
587
12
23
                if(Scalar::Util::blessed($logger)) {
588
4
14
                        $self->{'logger'} = $logger;
589                } else {
590
8
16
                        $self->{'logger'} = Log::Abstraction->new($logger);
591                }
592
12
143
                return $self;
593        }
594
1
16
        Carp::croak('Usage: set_logger(logger => $logger)')
595}
596
597# Open the database connection based on the specified type (e.g., SQLite, CSV).
598# Read the data into memory or establish a connection to the database file.
599# column_names allows the column names to be overridden on CSV files
600
601sub _open
602{
603        # Enforce that _open is only reachable from within this class hierarchy;
604        # caller() returns the calling package name as a plain string.
605
123
123
123
346
241
1417
        do { my $c = (caller)[0]; Carp::croak('Illegal Operation: _open may only be called within ', __PACKAGE__) unless $c && $c->isa(__PACKAGE__) };
606
607
122
116
        my $self = shift;
608
122
208
        my $params = Params::Get::get_params(undef, @_);
609
610
122
1910
        $params->{'sep_char'} ||= $self->{'sep_char'} ? $self->{'sep_char'} : '!';
611
122
253
        my $max_slurp_size = $params->{'max_slurp_size'} || $self->{'max_slurp_size'};
612
613
122
222
        my $table = $self->{'table'} || ref($self);
614
122
221
        $table =~ s/.*:://;
615
616
122
344
        $self->_trace(ref($self), ": _open $table");
617
618
122
4293
        return if($self->{$table});
619
620        # Read in the database
621
122
108
        my $dbh;
622
623        # DSN-based connection bypasses file detection entirely
624
122
357
        if(my $dsn = $self->{'dsn'} || $defaults{'dsn'}) {
625
14
78
                require DBI && DBI->import() unless DBI->can('connect');
626
627
14
22
                my $dialect = 'generic';
628
14
14
41
22
                if    ($dsn =~ /^dbi:SQLite:/i) { $dialect = 'sqlite'   }
629
0
0
                elsif ($dsn =~ /^dbi:Pg:/i)     { $dialect = 'postgres' }
630
0
0
                elsif ($dsn =~ /^dbi:mysql:/i)  { $dialect = 'mysql'    }
631
14
20
                $self->{'_dialect'} = $dialect;
632
633                $dbh = DBI->connect(
634                        $dsn,
635                        $self->{'username'},
636
14
73
                        $self->{'password'},
637                        { RaiseError => 1, AutoCommit => 1 },
638                ) or Carp::croak(ref($self), ": cannot connect: $DBI::errstr");
639
640
14
4060
                if($dialect eq 'sqlite') {
641
14
40
                        $dbh->do('PRAGMA synchronous = OFF');
642
14
215
                        $dbh->do('PRAGMA cache_size = -4096');
643
14
111
                        $dbh->do('PRAGMA journal_mode = OFF');
644
14
108
                        $dbh->do('PRAGMA temp_store = MEMORY');
645
14
90
                        $dbh->do('PRAGMA mmap_size = 1048576');
646
14
122
                        $dbh->sqlite_busy_timeout(100000);
647                }
648
649
14
55
                $self->{'type'} = 'DBI';
650
14
32
                $self->{$table} = $dbh;
651
14
29
                $self->{'_updated'} = time();
652
14
26
                return $self;
653        }
654
655
108
1802
        my $dir = Cwd::abs_path($self->{'directory'} || $defaults{'directory'});
656
108
245
        my $dbname = $self->{'dbname'} || $defaults{'dbname'} || $table;
657
108
402
        Carp::croak(ref($self), ": unsafe dbname '$dbname'")
658                unless $dbname =~ /^[a-zA-Z0-9_.-]+$/ && $dbname !~ /\.\./;
659
108
531
        my $slurp_file = File::Spec->catfile($dir, "$dbname.sql");
660
661
108
281
        $self->_debug("_open: try to open $slurp_file");
662
663        # Look at various places to find the file and derive the file type from the file's name
664
108
2826
        if(-r $slurp_file) {
665                # SQLite file
666
12
44
                require DBI && DBI->import() unless DBI->can('connect');
667
668
12
536
                require DBD::SQLite::Constants;
669
12
1101
                $dbh = DBI->connect("dbi:SQLite:dbname=$slurp_file", undef, undef, {
670                        sqlite_open_flags => DBD::SQLite::Constants::SQLITE_OPEN_READONLY(),
671                });
672        }
673
108
3425
        if($dbh) {
674
12
29
                $dbh->do('PRAGMA synchronous = OFF');
675
12
139
                $dbh->do('PRAGMA cache_size = -4096');       # Use 4MB cache - negative = KB)
676
12
91
                $dbh->do('PRAGMA journal_mode = OFF');       # Read-only, no journal needed
677
12
84
                $dbh->do('PRAGMA temp_store = MEMORY');      # Store temp data in RAM
678
12
69
                $dbh->do('PRAGMA mmap_size = 1048576');      # Use 1MB memory-mapped I/O
679
12
89
                $dbh->sqlite_busy_timeout(100000);   # 10s
680
12
30
                $self->_debug("read in $table from SQLite $slurp_file");
681
12
259
                $self->{'type'} = 'DBI';
682        } elsif($self->_is_berkeley_db(File::Spec->catfile($dir, "$dbname.db"))) {
683
0
0
                $self->_debug("$table is a BerkeleyDB file");
684
0
0
                $self->{'type'} = 'BerkeleyDB';
685        } else {
686
96
109
                my $fin;
687                # File::pfopen splits $path on ':' which breaks Windows drive letters
688                # (C:\foo becomes ['C', '\foo']).  Since we always have a single directory
689                # we use File::Spec->catfile directly — same behaviour, portable.
690
96
145
                for my $ext (qw(csv.gz db.gz)) {
691
192
662
                        my $candidate = File::Spec->catfile($dir, "$dbname.$ext");
692
192
1296
                        next unless -r $candidate;
693
0
0
                        open($fin, '<', $candidate) or next;
694
0
0
                        $slurp_file = $candidate;
695
0
0
                        last;
696                }
697
96
387
                if(defined($slurp_file) && (-r $slurp_file)) {
698
0
0
                        require Gzip::Faster;
699
0
0
                        Gzip::Faster->import();
700
701
0
0
                        close($fin);
702
0
0
                        $fin = File::Temp->new(SUFFIX => '.csv', UNLINK => 1);
703
0
0
                        print $fin gunzip_file($slurp_file);
704
0
0
                        $fin->flush();
705
0
0
                        $slurp_file = $fin->filename();
706
0
0
                        $self->{'_temp_fh'} = $fin;  # Keep object alive; auto-unlinks at DESTROY
707                } else {
708
96
307
                        my $psv = File::Spec->catfile($dir, "$dbname.psv");
709
96
416
                        if(-r $psv && open($fin, '<', $psv)) {
710                                # Pipe separated file
711
8
1982
                                $slurp_file = $psv;
712
8
12
                                $params->{'sep_char'} = '|';
713                        } else {
714                                # CSV or BerkeleyDB-extension file
715
88
96
                                for my $ext (qw(csv db)) {
716
102
280
                                        my $candidate = File::Spec->catfile($dir, "$dbname.$ext");
717
102
463
                                        next unless -r $candidate;
718
75
179
                                        open($fin, '<', $candidate) or next;
719
75
15987
                                        $slurp_file = $candidate;
720
75
88
                                        last;
721                                }
722                        }
723                }
724
96
280
                if(my $filename = $self->{'filename'} || $defaults{'filename'}) {
725
1
3
                        Carp::croak(ref($self), ": unsafe filename '$filename'")
726                                unless $filename =~ /^[a-zA-Z0-9_.-]+$/ && $filename !~ /\.\./;
727
1
2
                        $self->_debug("Looking for $filename in $dir");
728
1
17
                        $slurp_file = File::Spec->catfile($dir, $filename);
729                }
730
96
532
                if(defined($slurp_file) && (-r $slurp_file)) {
731
84
202
                        close($fin) if(defined($fin));
732
84
7594
                        my $sep_char = $params->{'sep_char'};
733
734
84
216
                        $self->_debug(__LINE__, ' of ', __PACKAGE__, ": slurp_file = $slurp_file, sep_char = $sep_char");
735
736
84
2059
                        if($params->{'column_names'}) {
737                                $dbh = DBI->connect("dbi:CSV:db_name=$slurp_file", undef, undef,
738                                        {
739                                                csv_sep_char => $sep_char,
740                                                csv_tables => {
741                                                        $table => {
742
1
7
                                                                col_names => $params->{'column_names'},
743                                                        },
744                                                },
745                                                f_dir      => $dir,
746                                                RaiseError => 1,
747                                                PrintError => 0
748                                        }
749                                );
750                        } else {
751
83
418
                                $dbh = DBI->connect("dbi:CSV:db_name=$slurp_file", undef, undef, { csv_sep_char => $sep_char, f_dir => $dir, RaiseError => 1 });
752                        }
753
84
1116154
                        $dbh->{'RaiseError'} = 1;
754
755
84
514
                        $self->_debug("read in $table from CSV $slurp_file");
756
757
84
2841
                        $dbh->{csv_tables}->{$table} = {
758                                allow_loose_quotes => 1,
759                                blank_is_undef => 1,
760                                empty_is_undef => 1,
761                                binary => 1,
762                                f_file => $slurp_file,
763                                escape_char => '\\',
764                                sep_char => $sep_char,
765                                # Don't do this, causes "Bizarre copy of HASH
766                                #       in scalar assignment in error_diag
767                                #       RT121127
768                                # auto_diag => 1,
769                                auto_diag => 0,
770                                # Don't do this, it causes "Attempt to free unreferenced scalar"
771                                # callbacks => {
772                                        # after_parse => sub {
773                                                # my ($csv, @rows) = @_;
774                                                # my @rc;
775                                                # foreach my $row(@rows) {
776                                                        # if($row->[0] !~ /^#/) {
777                                                                # push @rc, $row;
778                                                        # }
779                                                # }
780                                                # return @rc;
781                                        # }
782                                # }
783                        };
784
785                        # Text::xSV::Slurp cannot override column names, so skip slurp when
786                        # column_names is set — the DBI CSV connection will supply names instead.
787
84
62232
                        if(((-s $slurp_file) <= $max_slurp_size) && !$params->{'column_names'}) {
788
72
295
                                if((-s $slurp_file) == 0) {
789                                        # Empty file
790
0
0
                                        $self->{'data'} = ();
791                                } else {
792
72
3833
                                        require Text::xSV::Slurp;
793
72
58705
                                        Text::xSV::Slurp->import();
794
795
72
148
                                        $self->_debug('slurp in');
796
797
72
2291
                                        my $dataref = xsv_slurp(
798                                                shape => 'aoh',
799                                                text_csv => {
800                                                        sep_char => $sep_char,
801                                                        allow_loose_quotes => 1,
802                                                        blank_is_undef => 1,
803                                                        empty_is_undef => 1,
804                                                        binary => 1,
805                                                        escape_char => '\\',
806                                                },
807                                                # string => \join('', grep(!/^\s*(#|$)/, <DATA>))
808                                                file => $slurp_file
809                                        );
810
811                                        # Filter out blank lines and comment rows (lines starting with #)
812
72
311
335
72
13391
397
346
90
                                        my @data = grep { $_->{$self->{'id'}} !~ /^\s*#/ } grep { defined($_->{$self->{'id'}}) } @{$dataref};
813
814
72
130
                                        if($self->{'no_entry'}) {
815                                                # Not keyed on a primary column — keep as ordered list.
816                                                # Only store a reference when rows were found; an empty-array ref
817                                                # is truthy, which would activate the in-memory fast-path and
818                                                # silently return 0 results instead of falling through to SQL.
819
8
24
                                                $self->{'data'} = @data ? \@data : undef;
820                                        } else {
821                                                # Key the hash by $self->{'id'} for O(1) entry lookups
822
64
250
66
398
                                                $self->{'data'} = { map { $_->{$self->{'id'}} => $_ } @data };
823                                        }
824                                }
825                        }
826
84
233
                        $self->{'type'} = 'CSV';
827                } else {
828
12
41
                        $slurp_file = File::Spec->catfile($dir, "$dbname.xml");
829
12
62
                        if(-r $slurp_file) {
830
10
36
                                if((-s $slurp_file) <= $max_slurp_size) {
831
7
363
                                        require XML::Simple;
832
7
3828
                                        XML::Simple->import();
833
834
7
248
                                        my $xml = XMLin($slurp_file);
835
7
7
71620
11
                                        my @keys = keys %{$xml};
836
7
9
                                        my $key = $keys[0];
837
7
5
                                        my @data;
838
7
26
                                        if(ref($xml->{$key}) eq 'ARRAY') {
839
3
3
4
5
                                                @data = @{$xml->{$key}};
840                                        } elsif(ref($xml) eq 'ARRAY') {
841
0
0
0
0
                                                @data = @{$xml};
842                                        } elsif((ref($xml) eq 'HASH') && !$self->{'no_entry'}) {
843
3
3
3
4
                                                if(scalar(keys %{$xml}) == 1) {
844
2
3
                                                        if($xml->{$table}) {
845
1
2
                                                                @data = $xml->{$table};
846                                                        } else {
847
1
13
                                                                Carp::croak('XML slurp: complex documents with an "entry" field are not yet supported');
848                                                        }
849                                                } else {
850
1
6
                                                        Carp::croak('XML slurp: multi-key documents are not yet supported');
851                                                }
852                                        } else {
853
1
11
                                                Carp::croak('XML slurp: cannot handle ', ref($xml), ' structure');
854                                        }
855
4
8
                                        $self->{'data'} = ();
856
4
12
                                        if($self->{'no_entry'}) {
857                                                # Not keyed, will need to scan each entry
858
1
1
                                                my $i = 0;
859
1
2
                                                foreach my $d(@data) {
860
5
7
                                                        $self->{'data'}->{$i++} = $d;
861                                                }
862                                        } else {
863                                                # keyed on the $self->{'id'} (default: "entry") column
864
3
4
                                                foreach my $d(@data) {
865
11
19
                                                        $self->{'data'}->{$d->{$self->{'id'}}} = $d;
866                                                }
867                                        }
868                                } else {
869
3
11
                                        $dbh = DBI->connect('dbi:XMLSimple(RaiseError=>1):');
870
3
187673
                                        $dbh->{'RaiseError'} = 1;
871
3
38
                                        $self->_debug("read in $table from XML $slurp_file");
872
3
127
                                        $dbh->func($table, 'XML', $slurp_file, 'xmlsimple_import');
873                                }
874                        } else {
875                                # throw Error(-file => "$dir/$table");
876
2
9
                                $self->_fatal("Can't find a file called '$dbname' for the table $table in $dir");
877                        }
878
7
39
                        $self->{'type'} = 'XML';
879                }
880        }
881
882        # ref() must be called on the variable, not on the result of 'eq'
883
103
68
403
197
        Data::Reuse::fixate(%{$self->{'data'}}) if($self->{'data'} && (ref($self->{'data'}) eq 'HASH'));
884
885
103
10494
        $self->{$table} = $dbh;
886
103
625
        my @statb = stat($slurp_file);
887
103
192
        $self->{'_updated'} = $statb[9];
888
889
103
197
        return $self;
890}
891
892 - 928
=head2 selectall_arrayref

Returns a reference to an array of hash references for every row that
matches the given criteria, or C<undef> when there are no matches.

    my $rows = $db->selectall_arrayref();                    # all rows
    my $rows = $db->selectall_arrayref(status => 'active');  # exact match
    my $rows = $db->selectall_arrayref(score => { '>' => 8 });  # operator

The full criteria syntax is described in L</QUERY CRITERIA>.

Pass a C<join> key to combine with another table:

    my $rows = $db->selectall_arrayref(
        dept_name => 'Engineering',
        join      => { table => 'dept', on => 'e.dept_id = dept.id' },
    );

Results are returned in the cache (if configured) and the returned array
reference is made read-only unless C<no_fixate> was set.

B<Note:> this always returns all matching rows.  Use L</selectall_array>
in scalar context, or C<< $db->query->limit(1)->all() >>, to fetch just one row.

=head3 PSEUDOCODE

    1. Parse criteria; extract and build any JOIN clause.
    2. If data is slurped AND no joins AND criteria are simple:
       a. No criteria -> return all rows as arrayref.
       b. entry-only lookup -> return [$data{entry}].
       c. Otherwise -> scan rows in-memory with _match_criterion.
    3. Otherwise build SQL: SELECT * FROM table [JOIN] [WHERE] ORDER BY id.
    4. Check cache; return cached arrayref on HIT.
    5. prepare_cached + execute; fetch all rows.
    6. Store result in cache; fixate the array; return arrayref.

=cut
929
930sub selectall_arrayref {
931
117
36210
        my $self = shift;
932
933        # Fire _open() first so $self->{'berkeley'} is known before we parse @_.
934        # BerkeleyDB param parsing must use get_params(undef, \@_) so that
935        # key-value pairs like (join => {...}) are not mangled by the positional
936        # 'entry' mapping that non-BerkeleyDB paths use.
937
117
208
        $self->_open_table({});
938
939
117
102
        my $params;
940
941
117
150
        if($self->{'berkeley'}) {
942
1
3
                $params = Params::Get::get_params(undef, \@_) // {};
943
1
16
                return set_return($self->_scan_berkeley($params), { type => 'arrayref' });
944        }
945
946
116
172
        if($self->{'no_entry'}) {
947
75
115
                $params = Params::Get::get_params(undef, \@_);
948        } elsif(scalar(@_)) {
949
25
35
                $params = Params::Get::get_params('entry', @_);
950        }
951
952
116
1302
        my $table = $self->_open_table($params);
953
954
116
154
        $params //= {};
955
956
116
85
        my $join_clause = '';
957
116
141
        if(my $join_spec = delete $params->{'join'}) {
958
19
36
                $join_clause = $self->_build_joins($join_spec);
959        }
960
961
109
273
        if(!$join_clause && $self->{'data'} && !$self->_has_complex_criteria($params)) {
962
27
27
19
51
                if(scalar(keys %{$params}) == 0) {
963
11
35
                        $self->_trace("$table: selectall_arrayref fast track return");
964
11
271
                        if(ref($self->{'data'}) eq 'HASH') {
965
10
10
16
26
                                $self->_debug("$table: returning ", scalar keys %{$self->{'data'}}, ' entries');
966
10
10
206
22
                                if(scalar keys %{$self->{'data'}} <= 10) {
967
10
10
10
9
33
36
                                        $self->_debug(do { require Data::Dumper; Data::Dumper::Dumper($self->{'data'}) });
968                                }
969
10
10
204
21
                                my @rc = values %{$self->{'data'}};
970
10
30
                                return set_return(\@rc, { type => 'arrayref' });
971                        }
972
1
4
                        return set_return($self->{'data'}, { type => 'arrayref'});
973
16
51
                } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) {
974                        # exists() guard: fixate() locks all keys in the slurp hash; return []
975                        # (not [undef]) when the key is missing so callers get an empty result
976                        return set_return([], { type => 'arrayref' })
977
7
22
                                unless exists($self->{'data'}->{$params->{'entry'}});
978
5
16
                        return set_return([$self->{'data'}->{$params->{'entry'}}], { type => 'arrayref' });
979                } elsif(ref($self->{'data'}) eq 'HASH') {
980                        # Scan in-memory hash for simple column criteria without touching DBI.
981                        # fixate() locks hash keys, so use exists() to avoid throwing on unknown columns.
982
9
21
                        $self->_debug("$table: selectall_arrayref in-memory scan with criteria");
983                        my @rc = grep {
984
36
26
                                my $row = $_;
985
36
36
36
47
65
43
                                all { $self->_match_criterion(exists($row->{$_}) ? $row->{$_} : undef, $params->{$_}) } keys %{$params}
986
9
9
185
13
                        } values %{$self->{'data'}};
987
9
19
                        return set_return(\@rc, { type => 'arrayref' });
988                }
989        }
990
991
82
186
        my ($where, $wargs) = $self->_build_where($params);
992
76
76
61
69
        my @query_args = @{$wargs};
993
994
76
59
        my $query = "SELECT * FROM $table";
995
76
83
        $query .= " $join_clause" if $join_clause;
996
76
166
        if($join_clause) {
997
11
12
                $query .= " WHERE $where" if $where;
998        } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
999
6
7
                my $id = $self->{'id'};
1000
6
10
                $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1001
6
10
                $query .= " AND ($where)" if $where;
1002        } else {
1003
59
63
                $query .= " WHERE $where" if $where;
1004        }
1005
76
101
        if(!$self->{'no_entry'}) {
1006
10
13
                $query .= ' ORDER BY ' . $self->{'id'};
1007        }
1008
1009
76
72
        if(defined($query_args[0])) {
1010
50
108
                $self->_debug("selectall_arrayref $query: ", join(', ', @query_args));
1011        } else {
1012
26
47
                $self->_debug("selectall_arrayref $query");
1013        }
1014
1015
76
1611
        my $key;
1016        my $c;
1017
76
88
        if($c = $self->{cache}) {
1018
0
0
                $key = ref($self) . "::$query array";
1019
0
0
                if(defined($query_args[0])) {
1020
0
0
                        $key .= ' ' . join(', ', @query_args);
1021                }
1022
0
0
                $self->_debug("cache key = '$key'");
1023
0
0
                if(my $rc = $c->get($key)) {
1024
0
0
                        $self->_debug('cache HIT');
1025
0
0
                        return $rc;     # We stored a ref to the array
1026
1027                        # This use of a temporary variable is to avoid
1028                        #       "Implicit scalar context for array in return"
1029                        # my @rc = @{$rc};
1030                        # return @rc;
1031                }
1032
0
0
                $self->_debug('cache MISS');
1033        } else {
1034
76
85
                $self->_debug('cache not used');
1035        }
1036
1037
76
1344
        if(my $sth = $self->{$table}->prepare_cached($query)) {
1038
76
16152
                $sth->execute(@query_args) || croak("$query: @query_args");
1039
1040
76
24359
                my $rc;
1041
76
703
                while(my $href = $sth->fetchrow_hashref()) {
1042
205
205
205
1369
968
217
                        push @{$rc}, $href if(scalar keys %{$href});
1043                }
1044
76
330
                $c->set($key, $rc, $self->{'cache_duration'}) if $c;
1045
1046
76
98
                if(!$self->{'no_fixate'}) {
1047                        # forget() clears stale address→canonical mappings from prior calls;
1048                        # fixate() then deduplicates values within this result set only.
1049                        # Without forget(), freed hashref addresses from previous fixate calls
1050                        # can collide with new DBI hashrefs and return wrong canonical rows.
1051
76
136
                        Data::Reuse::forget();
1052
76
76
580
94
                        Data::Reuse::fixate(@{$rc});
1053                }
1054
1055
76
11881
                return $rc;
1056        }
1057
0
0
        $self->_warn("selectall_arrayref failure on $query: @query_args");
1058
0
0
        croak("$query: @query_args");
1059}
1060
1061 - 1066
=head2 selectall_hashref

Deprecated alias for L</selectall_arrayref>.  Use C<selectall_arrayref> in
new code.

=cut
1067
1068sub selectall_hashref
1069{
1070
7
1261
        my $self = shift;
1071
7
542
        return $self->selectall_arrayref(@_);
1072}
1073
1074 - 1087
=head2 selectall_array

Similar to L</selectall_arrayref> but returns a list of hash references
rather than a reference to an array.

    my @rows = $db->selectall_array(status => 'active');

In B<scalar context> it applies C<LIMIT 1> and returns just the first
matching hash reference - making it more efficient than C<selectall_arrayref>
when you only need one row.  In B<list context> all matching rows are returned.

Accepts the same criteria and C<join> parameter as L</selectall_arrayref>.

=cut
1088
1089sub selectall_array
1090{
1091
21
2951
        my $self = shift;
1092
1093
21
60
        $self->_open_table({});
1094
1095
21
59
        if($self->{'berkeley'}) {
1096
1
2
                my $params = Params::Get::get_params(undef, \@_) // {};
1097
1
11
                my $rows = $self->_scan_berkeley($params);
1098
1
1
2
2
                return wantarray ? @{$rows} : $rows->[0];
1099        }
1100
1101
20
42
        my $params = Params::Get::get_params(undef, \@_);
1102
20
260
        my $table = $self->_open_table($params);
1103
1104
20
43
        $params //= {};
1105
20
48
        my $join_clause = '';
1106
20
39
        if(my $join_spec = delete $params->{'join'}) {
1107
1
2
                $join_clause = $self->_build_joins($join_spec);
1108        }
1109
1110
20
68
        if(!$join_clause && $self->{'data'} && !$self->_has_complex_criteria($params)) {
1111
14
14
12
25
                if(scalar(keys %{$params}) == 0) {
1112
8
19
                        $self->_trace("$table: selectall_array fast track return");
1113
8
161
                        if(ref($self->{'data'}) eq 'HASH') {
1114
7
7
8
19
                                return values %{$self->{'data'}};
1115                        }
1116
1
1
1
3
                        return @{$self->{'data'}};
1117
6
34
                } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) {
1118                        # exists() guard: fixate() locks all keys; return empty list (not undef)
1119                        # for a missing entry so callers in list context get 0 elements not 1
1120
2
7
                        return () unless exists($self->{'data'}->{$params->{'entry'}});
1121
1
3
                        return $self->{'data'}->{$params->{'entry'}};
1122                } elsif(ref($self->{'data'}) eq 'HASH') {
1123                        # Same as selectall_arrayref scan but returns a list
1124
4
9
                        $self->_debug("$table: selectall_array in-memory scan with criteria");
1125                        my @rc = grep {
1126
16
9
                                my $row = $_;
1127
16
16
16
25
30
18
                                all { $self->_match_criterion(exists($row->{$_}) ? $row->{$_} : undef, $params->{$_}) } keys %{$params}
1128
4
4
87
6
                        } values %{$self->{'data'}};
1129
4
11
                        return @rc;
1130                }
1131        }
1132
1133
6
15
        my ($where, $wargs) = $self->_build_where($params);
1134
6
6
6
9
        my @query_args = @{$wargs};
1135
1136
6
8
        my $query = "SELECT * FROM $table";
1137
6
10
        $query .= " $join_clause" if $join_clause;
1138
6
17
        if($join_clause) {
1139
1
2
                $query .= " WHERE $where" if $where;
1140        } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
1141
1
1
                my $id = $self->{'id'};
1142
1
2
                $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1143
1
1
                $query .= " AND ($where)" if $where;
1144        } else {
1145
4
11
                $query .= " WHERE $where" if $where;
1146        }
1147
6
10
        if(!$self->{'no_entry'}) {
1148
4
6
                $query .= ' ORDER BY ' . $self->{'id'};
1149        }
1150
6
10
        if(!wantarray) {
1151
2
1
                $query .= ' LIMIT 1';
1152        }
1153
1154
6
9
        if(defined($query_args[0])) {
1155
2
6
                $self->_debug("selectall_array $query: ", join(', ', @query_args));
1156        } else {
1157
4
7
                $self->_debug("selectall_array $query");
1158        }
1159
1160
6
154
        my $key;
1161        my $c;
1162
6
25
        if($c = $self->{cache}) {
1163
0
0
                $key = ref($self) . '::' . $query;
1164
0
0
                if(wantarray) {
1165
0
0
                        $key .= ' array';
1166                }
1167
0
0
                if(defined($query_args[0])) {
1168
0
0
                        $key .= ' ' . join(', ', @query_args);
1169                }
1170
0
0
                $self->_debug("cache key = '$key'");
1171
0
0
                if(my $rc = $c->get($key)) {
1172
0
0
                        $self->_debug('cache HIT');
1173
0
0
0
0
                        return wantarray ? @{$rc} : $rc;        # We stored a ref to the array
1174
1175                        # This use of a temporary variable is to avoid
1176                        #       "Implicit scalar context for array in return"
1177                        # my @rc = @{$rc};
1178                        # return @rc;
1179                }
1180
0
0
                $self->_debug('cache MISS');
1181        } else {
1182
6
9
                $self->_debug('cache not used');
1183        }
1184
1185
6
120
        if(my $sth = $self->{$table}->prepare_cached($query)) {
1186
6
3071
                $sth->execute(@query_args) || croak("$query: @query_args");
1187
1188
6
6512
                my $rc;
1189
6
60
                while(my $href = $sth->fetchrow_hashref()) {
1190
26
511
                        if(!wantarray) {
1191                                # Scalar context: return just the first row; cache it too
1192
2
6
                                $sth->finish();
1193
2
2
                                $c->set($key, [$href], $self->{'cache_duration'}) if $c;
1194
2
6
                                return $href;
1195                        }
1196
24
24
11
68
                        push @{$rc}, $href;
1197                }
1198
4
63
                $c->set($key, $rc, $self->{'cache_duration'}) if $c;
1199
1200
4
5
                if($rc) {
1201
4
8
                        if(!$self->{'no_fixate'}) {
1202
4
8
                                Data::Reuse::forget();
1203
4
4
22
7
                                Data::Reuse::fixate(@{$rc});
1204                        }
1205
4
4
966
16
                        return @{$rc};
1206                }
1207
0
0
                return;
1208        }
1209
0
0
        $self->_warn("selectall_array failure on $query: @query_args");
1210
0
0
        croak("$query: @query_args");
1211}
1212
1213 - 1218
=head2 selectall_hash

Deprecated alias for L</selectall_array>.  Use C<selectall_array> in new
code.

=cut
1219
1220sub selectall_hash
1221{
1222
8
2309
        my $self = shift;
1223
8
50
        return $self->selectall_array(@_);
1224}
1225
1226 - 1236
=head2 count

Returns the number of rows matching the given criteria.

    my $total  = $db->count();
    my $active = $db->count(status => 'active');
    my $high   = $db->count(score  => { '>' => 90 });

Accepts the full criteria syntax described in L</QUERY CRITERIA>.

=cut
1237
1238sub count
1239{
1240
76
6708
        my $self = shift;
1241
1242
76
164
        $self->_open_table({});
1243
1244
72
145
        if($self->{'berkeley'}) {
1245
1
2
                my $params = Params::Get::get_params(undef, \@_) // {};
1246
1
1
10
3
                return scalar @{$self->_scan_berkeley($params)};
1247        }
1248
1249
71
147
        my $params = Params::Get::get_params(undef, \@_);
1250
71
879
        my $table = $self->_open_table($params);
1251
1252
71
146
        if($self->{'data'}) {
1253
43
43
34
70
                if(scalar(keys %{$params}) == 0) {
1254
31
69
                        $self->_trace("$table: count fast track return");
1255
31
744
                        if(ref($self->{'data'}) eq 'HASH') {
1256
28
28
21
74
                                return scalar keys %{$self->{'data'}};
1257                        }
1258
3
3
3
10
                        return scalar @{$self->{'data'}};
1259
12
64
                } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) {
1260                        # exists() guard: fixate() locks all keys in the slurp hash
1261
7
37
                        return (exists($self->{'data'}->{$params->{'entry'}}) && $self->{'data'}->{$params->{'entry'}}) ? 1 : 0;
1262                }
1263        }
1264
1265
33
92
        my ($where, $wargs) = $self->_build_where($params);
1266
29
29
26
36
        my @query_args = @{$wargs};
1267
1268
29
25
        my $query;
1269
29
97
        if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
1270
12
16
                my $id = $self->{'id'};
1271
12
19
                $query = "SELECT COUNT(*) FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1272
12
24
                $query .= " AND ($where)" if $where;
1273        } elsif($self->{'no_entry'}) {
1274
13
17
                $query = "SELECT COUNT(*) FROM $table";
1275
13
24
                $query .= " WHERE $where" if $where;
1276        } else {
1277
4
9
                $query = "SELECT COUNT(" . $self->{'id'} . ") FROM $table";
1278
4
8
                $query .= " WHERE $where" if $where;
1279        }
1280
1281
29
49
        if(defined($query_args[0])) {
1282
11
28
                $self->_debug("count $query: ", join(', ', @query_args));
1283        } else {
1284
18
41
                $self->_debug("count $query");
1285        }
1286
1287
29
624
        my $key;
1288        my $c;
1289
29
54
        if($c = $self->{'cache'}) {
1290                # Opportunistic: if a selectall_arrayref for the same criteria is already
1291                # in cache, derive the count from that array rather than hitting the DB.
1292                # The key is built to match what selectall_arrayref would store.
1293
0
0
                $key = ref($self) . '::' . $query;
1294
0
0
                $key =~ s/COUNT\((.+?)\)/$1/;
1295
0
0
                $key .= ' array';
1296
0
0
                if(defined($query_args[0])) {
1297
0
0
                        $key .= ' ' . join(', ', @query_args);
1298                }
1299
0
0
                if(my $rc = $c->get($key)) {
1300
0
0
                        $self->_debug('count: cache HIT (selectall array)');
1301
0
0
0
0
                        return ref($rc) eq 'ARRAY' ? scalar @{$rc} : 0;
1302                }
1303
0
0
                $self->_debug('count: cache MISS');
1304        } else {
1305
29
37
                $self->_debug('cache not used');
1306        }
1307
1308
29
516
        if(my $sth = $self->{$table}->prepare_cached($query)) {
1309
29
30438
                $sth->execute(@query_args) || croak("$query: @query_args");
1310
1311
29
19069
                my $count = $sth->fetchrow_arrayref()->[0];
1312
29
424
                $sth->finish();
1313
1314
29
129
                return $count;
1315        }
1316
0
0
        $self->_warn("count failure on $query: @query_args");
1317
0
0
        croak("$query: @query_args");
1318}
1319
1320 - 1344
=head2 fetchrow_hashref

Returns a hash reference for the first row matching the given criteria,
or C<undef> when there is no match.  Always applies C<LIMIT 1>.

    my $row = $db->fetchrow_hashref(entry => 'key1');
    my $row = $db->fetchrow_hashref(score => { '>=' => 10 });

When C<no_entry> is B<not> set you may pass a single bare value and it is
used as the C<entry> key:

    my $row = $db->fetchrow_hashref('key1');    # same as entry => 'key1'

Accepts the full criteria syntax described in L</QUERY CRITERIA>, including
the C<join> parameter:

    my $row = $db->fetchrow_hashref(
        name => 'Alice',
        join => { table => 'dept', on => 'e.dept_id = dept.id' },
    );

Pass C<< table => $other_table >> to query a table other than the one
derived from the class name.

=cut
1345
1346sub fetchrow_hashref {
1347
44
11321
        my $self = shift;
1348
1349
44
127
        $self->_trace('Entering fetchrow_hashref');
1350
1351
44
1014
        my $params;
1352
1353
44
80
        if(!$self->{'no_entry'}) {
1354
35
70
                $params = Params::Get::get_params('entry', @_);
1355        } else {
1356
9
19
                $params = Params::Get::get_params(undef, @_);
1357        }
1358
1359
44
669
        my $table = $self->_open_table($params);
1360
1361        # ::diag($self->{'type'});
1362
44
31
131
146
        if($self->{'data'} && (!$self->{'no_entry'}) && (scalar keys(%{$params}) == 1) && defined($params->{'entry'}) && !$self->_has_complex_criteria($params)) {
1363
25
38
                $self->_debug('Fast return from slurped data');
1364                # Use exists() — fixate() locks the outer hash; accessing a missing key throws
1365
25
605
                return exists($self->{'data'}->{$params->{'entry'}}) ? $self->{'data'}->{$params->{'entry'}} : undef;
1366        }
1367
1368
19
38
        if($self->{'berkeley'}) {
1369                # print STDERR ">>>>>>>>>>>>\n";
1370                # ::diag(Data::Dumper->new([$self->{'berkeley'}])->Dump());
1371
0
0
0
0
                if((!$self->{'no_entry'}) && (scalar keys(%{$params}) == 1) && defined($params->{'entry'})) {
1372
0
0
                        return { entry => $self->{'berkeley'}->{$params->{'entry'}} };
1373                }
1374
0
0
                my $id = $self->{'id'};
1375
0
0
0
0
                if($self->{'no_entry'} && (scalar keys(%{$params}) == 1) && defined($id) && defined($params->{$id})) {
1376
0
0
                        if(my $rc = $self->{'berkeley'}->{$params->{$id}}) {
1377
0
0
                                return { $params->{$id} => $rc }  # Return key->value as a hash pair
1378                        }
1379
0
0
                        return;
1380                }
1381
0
0
                Carp::croak(ref($self), ': fetchrow_hashref is meaningless on a NoSQL database');
1382        }
1383
1384
19
54
        my $target = delete($params->{'table'}) // $table;
1385
19
22
        my $join_spec = delete $params->{'join'};
1386
19
40
        my $join_clause = $join_spec ? $self->_build_joins($join_spec) : '';
1387
19
47
        my ($where, $wargs) = $self->_build_where($params);
1388
17
17
15
22
        my @query_args = @{$wargs};
1389
1390
17
29
        my $query = "SELECT * FROM $target";
1391
17
22
        $query .= " $join_clause" if $join_clause;
1392
17
67
        if($join_clause) {
1393
1
1
                $query .= " WHERE $where" if $where;
1394        } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
1395
5
6
                my $id = $self->{'id'};
1396
5
5
                $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1397
5
10
                $query .= " AND ($where)" if $where;
1398        } else {
1399
11
43
                $query .= " WHERE $where" if $where;
1400        }
1401
17
25
        $query .= ' LIMIT 1';
1402
17
55
        if(defined($query_args[0])) {
1403
16
27
                my @call_details = caller(0);
1404
16
166
                $self->_debug("fetchrow_hashref $query: ", join(', ', @query_args),
1405                        ' called from ', $call_details[2], ' of ', $call_details[1]);
1406        } else {
1407
1
2
                $self->_debug("fetchrow_hashref $query");
1408        }
1409
17
324
        my $key = ref($self) . '::';
1410
17
65
        if(defined($query_args[0])) {
1411
16
24
                if(wantarray) {
1412
0
0
                        $key .= 'array ';
1413                }
1414
16
33
                $key .= "fetchrow $query " . join(', ', @query_args);
1415        } else {
1416
1
0
                $key .= "fetchrow $query";
1417        }
1418
17
16
        my $c;
1419
17
32
        if($c = $self->{cache}) {
1420
0
0
                if(my $rc = $c->get($key)) {
1421
0
0
                        if(wantarray) {
1422
0
0
                                if(ref($rc) eq 'ARRAY') {
1423
0
0
0
0
                                        return @{$rc};  # We stored a ref to the array
1424                                }
1425                        } else {
1426
0
0
                                return $rc;
1427                        }
1428                }
1429        }
1430
1431        my $sth = $self->{$table}->prepare_cached($query)
1432
17
86
                or Carp::croak(ref($self), ": prepare failed: ", $self->{$table}->errstr());
1433
17
20731
        $sth->execute(@query_args) || croak("$query: @query_args");
1434
17
9335
        my $rc = $sth->fetchrow_hashref();
1435
17
376
        $sth->finish();
1436
17
49
        if($c) {
1437
0
0
                if($rc) {
1438
0
0
                        $self->_debug("stash $key=>$rc in the cache for ", $self->{'cache_duration'});
1439
0
0
0
0
0
0
                        $self->_debug("returns ", do { require Data::Dumper; Data::Dumper->new([$rc])->Dump() });
1440                } else {
1441
0
0
                        $self->_debug("Stash $key=>undef in the cache for ", $self->{'cache_duration'});
1442                }
1443
0
0
                $c->set($key, $rc, $self->{'cache_duration'});
1444        }
1445
17
51
        return $rc;
1446}
1447
1448 - 1469
=head2 execute

Execute a raw SQL query on the underlying database.

    # Scalar context: returns the first row as a hashref
    my $row = $db->execute(query => 'SELECT * FROM foo WHERE id = 1');

    # List context: returns all rows as a list of hashrefs
    my @rows = $db->execute(query => 'SELECT * FROM foo WHERE score > ?',
                            args  => [80]);

The C<FROM E<lt>tableE<gt>> clause is appended automatically if omitted.

On CSV tables without C<no_entry> it may help to add
C<WHERE entry IS NOT NULL AND entry NOT LIKE '#%'> to filter comment rows.

If the data have been slurped into memory this method still hits the actual
database file directly.

C<args> is an arrayref of bind values (see L<DBI/execute>).

=cut
1470
1471sub execute
1472{
1473
18
4405
        my $self = shift;
1474
1475
18
39
        if($self->{'berkeley'}) {
1476
1
13
                Carp::croak(ref($self), ': execute is meaningless on a NoSQL database');
1477        }
1478
1479
17
30
        my $args = Params::Get::get_params('query', @_);
1480
1481        # Ensure the 'query' parameter is provided
1482        Carp::croak(__PACKAGE__, ': Usage: execute(query => $query)')
1483
15
190
                unless defined $args->{'query'};
1484
1485
15
28
        my $table = $self->_open_table($args);
1486
1487
15
18
        my $query = $args->{'query'};
1488
1489        # Append "FROM <table>" if missing
1490
15
38
        $query .= " FROM $table" unless $query =~ /\sFROM\s/i;
1491
1492        # Log the query if a logger is available
1493
15
34
        $self->_debug("execute $query");
1494
1495        # Prepare and execute the query
1496
15
364
        my $sth = $self->{$table}->prepare_cached($query);
1497        # DBI->execute() takes a list; normalise args to an array whether it
1498        # was passed as an arrayref ([30]) or a bare scalar/list (30).
1499
15
2342
        if(exists($args->{'args'})) {
1500
9
5
18
6
                my @bind = ref($args->{'args'}) eq 'ARRAY' ? @{$args->{'args'}} : ($args->{'args'});
1501
9
157
                $sth->execute(@bind) or croak("$query: ", join(', ', @bind));
1502        } else {
1503
6
110
                $sth->execute() or croak($query);
1504        }
1505
1506        # Fetch the results
1507
15
2972
        my @results;
1508
15
126
        while (my $row = $sth->fetchrow_hashref()) {
1509
31
263
                unless(wantarray) {
1510
3
11
                        $sth->finish();
1511
3
7
                        return $row;
1512                }
1513
28
118
                push @results, $row;
1514        }
1515
1516        # Return all rows as an array in list context
1517
12
88
        return @results;
1518}
1519
1520 - 1526
=head2 updated

Returns the Unix timestamp of the last database update (mtime for
file-based backends, or the time of the most recent C<new()> call for
DSN-based connections).

=cut
1527
1528sub updated {
1529
4
564
        my $self = shift;
1530
1531
4
11
        return $self->{'_updated'};
1532}
1533
1534 - 1555
=head2 columns

Returns an array reference of column names for the current table.

    my $cols = $db->columns();    # e.g. ['entry', 'name', 'score', 'status']

The column list is determined by the backend:

=over 4

=item * B<Slurp mode> - sorted keys of the first row in memory.

=item * B<SQLite / other DBI> - a zero-row C<SELECT *> exposes the driver's
C<NAME> attribute.

=item * B<BerkeleyDB> - always returns C<['entry', 'value']>.

=back

The result is cached inside the object after the first call.

=cut
1556
1557sub columns {
1558
16
2595
        my $self = shift;
1559
1560
16
55
        return $self->{'_columns'} if $self->{'_columns'};
1561
1562
12
36
        my $table = $self->_open_table({});
1563
1564
12
16
        my @cols;
1565
1566
12
21
        if($self->{'berkeley'}) {
1567
1
2
                return $self->{'_columns'} = ['entry', 'value'];
1568        }
1569
1570
11
23
        if(my $data = $self->{'data'}) {
1571
7
19
                if(ref($data) eq 'HASH') {
1572
7
7
7
11
                        my ($first) = values %{$data};
1573
7
7
11
13
                        @cols = sort keys %{$first} if $first;
1574                }
1575        } else {
1576
4
24
                my $sth = $self->{$table}->prepare_cached("SELECT * FROM $table WHERE 1=0");
1577
4
3637
                $sth->execute();
1578
4
4
6214
22
                @cols = @{$sth->{NAME}};
1579
4
34
                $sth->finish();
1580        }
1581
1582
11
45
        return $self->{'_columns'} = \@cols;
1583}
1584
1585 - 1628
=head2 schema

Returns a hash reference describing the schema of the current table.
Each key is a column name; each value is a hash reference with these keys:

=over 4

=item * C<type> - data type string (e.g. C<TEXT>, C<INTEGER>, C<REAL>)

=item * C<nullable> - C<1> if the column may be NULL, C<0> if NOT NULL

=item * C<default> - default value string, or C<undef>

=item * C<pk> - C<1> if this column is (part of) the primary key, C<0> otherwise

=back

    my $schema = $db->schema();

    for my $col (sort keys %{$schema}) {
        my $info = $schema->{$col};
        printf "%s  %s  %s\n",
            $col,
            $info->{type},
            $info->{nullable} ? 'NULL' : 'NOT NULL';
    }

The schema is determined by the backend:

=over 4

=item * B<SQLite> - C<PRAGMA table_info(table)>

=item * B<Other DBI drivers> - C<< $dbh->column_info(...) >>

=item * B<Slurp mode> - inferred from the first row (all columns typed as C<TEXT>)

=item * B<BerkeleyDB> - always returns C<entry> (pk) and C<value>

=back

The result is cached inside the object after the first call.

=cut
1629
1630sub schema {
1631
15
3817
        my $self = shift;
1632
1633
15
61
        return $self->{'_schema'} if $self->{'_schema'};
1634
1635
11
23
        my $table = $self->_open_table({});
1636
11
16
        my %schema;
1637
1638
11
21
        if($self->{'berkeley'}) {
1639
1
4
                return $self->{'_schema'} = {
1640                        entry => { type => 'TEXT', nullable => 0, default => undef, pk => 1 },
1641                        value => { type => 'TEXT', nullable => 1, default => undef, pk => 0 },
1642                };
1643        }
1644
1645
10
21
        if(my $data = $self->{'data'}) {
1646
6
14
                if(ref($data) eq 'HASH') {
1647
6
6
7
10
                        my ($first) = values %{$data};
1648
6
18
                        if($first) {
1649
6
8
                                my $id = $self->{'id'};
1650
6
6
6
15
                                for my $col (keys %{$first}) {
1651
12
42
                                        $schema{$col} = {
1652                                                type     => 'TEXT',
1653                                                nullable => ($col eq $id ? 0 : 1),
1654                                                default  => undef,
1655                                                pk       => ($col eq $id ? 1 : 0),
1656                                        };
1657                                }
1658                        }
1659                }
1660        } else {
1661
4
40
                my $driver = $self->{$table}->{'Driver'}{'Name'} // '';
1662
4
23
                if($driver eq 'SQLite') {
1663
3
11
                        my $sth = $self->{$table}->prepare_cached("PRAGMA table_info($table)");
1664
3
163
                        $sth->execute();
1665
3
47
                        while(my $row = $sth->fetchrow_hashref()) {
1666                                $schema{$row->{'name'}} = {
1667                                        type     => $row->{'type'},
1668                                        nullable => !$row->{'notnull'},
1669                                        default  => $row->{'dflt_value'},
1670
8
72
                                        pk       => $row->{'pk'},
1671                                };
1672                        }
1673
3
36
                        $sth->finish();
1674                } else {
1675
1
12
                        my $sth = $self->{$table}->column_info(undef, undef, $table, '%');
1676
1
2
                        if($sth) {
1677
0
0
                                while(my $row = $sth->fetchrow_hashref()) {
1678                                        $schema{$row->{'COLUMN_NAME'}} = {
1679                                                type     => $row->{'TYPE_NAME'},
1680                                                nullable => $row->{'NULLABLE'},
1681
0
0
                                                default  => $row->{'COLUMN_DEF'},
1682                                                pk       => 0,
1683                                        };
1684                                }
1685
0
0
                                $sth->finish();
1686                        }
1687                }
1688        }
1689
1690
10
33
        return $self->{'_schema'} = \%schema;
1691}
1692
1693 - 1714
=head2 query

Returns a new L<Database::Abstraction::Query> builder object bound to this
database instance, for fluent method-chaining queries.

    # All active rows with high scores, newest first, max 10
    my $rows = $db->query
        ->where(status => 'active')
        ->where(score  => { '>' => 80 })
        ->order_by('score DESC')
        ->limit(10)
        ->all();

    # Single row
    my $row = $db->query->where(name => 'Alice')->first();

    # Just a count
    my $n = $db->query->where(status => 'active')->count();

See L<Database::Abstraction::Query> for the full API.

=cut
1715
1716sub query
1717{
1718
57
17003
        my $self = shift;
1719
57
789
        require Database::Abstraction::Query;
1720
57
103
        return Database::Abstraction::Query->new(_db => $self);
1721}
1722
1723 - 1767
=head2 AUTOLOAD - column shortcut

Calling an unknown method whose name matches a column name performs a column
lookup.  The method name is the column you want; the arguments are criteria.

    # Scalar context: return the first match
    my $name = $db->name(entry => 'key1');

    # List context: return all matching values
    my @names = $db->name();

    # Shortcut when the table has an 'entry' key column
    my $name = $db->name('key1');    # same as name(entry => 'key1')

    # Unique/distinct values
    my @statuses = $db->status(distinct => 1);

B<In list context> the full column is returned (all rows), ordered by the
column value.  B<In scalar context> only the first match is returned
(C<LIMIT 1>).

Results come from the slurp cache when available.

Throws an error if the column does not exist (slurp mode) or if AUTOLOAD
has been disabled with C<< auto_load => 0 >>.

=head3 PSEUDOCODE

    1. Extract column name from $AUTOLOAD; guard on DESTROY.
    2. Croak if auto_load => 0.
    3. Validate $column against /^[a-zA-Z_][a-zA-Z0-9_]*$/.
    4. If data is slurped:
       a. List context, no params -> map column over all rows (exists guard).
       b. entry-only param -> direct hash lookup (exists guard).
       c. No params, scalar -> first value in hash.
       d. no_entry set -> scan array for matching key/value pair.
       e. Other params -> scan keyed hash for matching column.
    5. If not slurped, build SQL:
       - List:   SELECT column FROM table [WHERE ...] ORDER BY column
       - Scalar: SELECT DISTINCT column FROM table [WHERE ...] LIMIT 1
    6. Check cache; return on HIT.
    7. prepare_cached + execute; fetch result.
    8. Store in cache; fixate; return.

=cut
1768
1769sub AUTOLOAD {
1770
82
12972
        our $AUTOLOAD;
1771
82
331
        my ($column) = $AUTOLOAD =~ /::(\w+)$/;
1772
1773
82
141
        return if($column eq 'DESTROY');
1774
1775
82
144
        my $self = shift or return;
1776
1777
82
110
        Carp::croak(__PACKAGE__, ": Unknown column $column") if(!ref($self));
1778
1779        # Allow the AUTOLOAD feature to be disabled
1780
82
783
        Carp::croak(__PACKAGE__, ": AUTOLOAD disabled (auto_load => 0)") if(exists($self->{'auto_load'}) && !$self->{'auto_load'});
1781
1782        # Validate column name - only allow safe column name
1783
78
195
        Carp::croak(__PACKAGE__, ": Invalid column name: $column") unless $column =~ /^[a-zA-Z_][a-zA-Z0-9_]*$/;
1784
1785
78
1353
        my $table = $self->_open_table();
1786
1787
77
61
        my %params;
1788
77
171
        if(ref($_[0]) eq 'HASH') {
1789
2
2
2
5
                %params = %{$_[0]};
1790        } elsif((scalar(@_) % 2) == 0) {
1791
53
81
                %params = @_;
1792        } elsif(scalar(@_) == 1) {
1793                # Don't error on key-value databases, since there's no idea of columns
1794
22
58
                if($self->{'no_entry'} && !$self->{'berkeley'}) {
1795
0
0
                        Carp::croak(ref($self), "::($_[0]): ", $self->{'id'}, ' is not a column');
1796                }
1797
22
31
                $params{'entry'} = shift;
1798        }
1799
1800
77
122
        if($self->{'berkeley'}) {
1801
0
0
                if(my $id = $self->{'id'}) {
1802
0
0
                        return $self->{'berkeley'}->{$params{$id}};
1803                }
1804
0
0
                return $self->{'berkeley'}->{$params{'entry'}};
1805        }
1806
1807
77
109
        croak('Where did the data come from?') if(!defined($self->{'type'}));
1808
77
60
        my $query;
1809
77
82
        my $done_where = 0;
1810
77
171
        my $distinct = delete($params{'distinct'}) || delete($params{'unique'});
1811
1812
77
129
        if(wantarray && !$distinct) {
1813
18
53
                if(((scalar keys %params) == 0) && (my $data = $self->{'data'})) {
1814                        # Return all column values from the in-memory hash.
1815                        # Use exists() because fixate() locks inner row hashes —
1816                        # accessing a disallowed key would throw without the guard.
1817                        # Handle both HASH (keyed data) and ARRAY (no_entry CSV slurp).
1818
15
2
13
54
2
21
                        my @_rows = ref($data) eq 'ARRAY' ? @{$data} : values %{$data};
1819
15
62
20
94
                        return map { exists($_->{$column}) ? $_->{$column} : undef } @_rows;
1820                }
1821
3
4
                my $id = $self->{'id'};
1822
3
9
                if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
1823
2
3
                        $query = "SELECT $column FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1824
2
3
                        $done_where = 1;
1825                } else {
1826
1
2
                        $query = "SELECT $column FROM $table";
1827                }
1828        } else {
1829
59
98
                if(my $data = $self->{'data'}) {
1830                        # The data has been read in using Text::xSV::Slurp,
1831                        #       so no need to do any SQL
1832
42
59
                        $self->_debug('AUTOLOAD using slurped data');
1833
42
1074
                        if($self->{'no_entry'}) {
1834
2
3
                                $self->_debug('no_entry is set');
1835
2
24
                                my ($key, $value) = %params;
1836
2
4
                                if(defined($key)) {
1837
2
5
                                        $self->_debug("key = $key, value = $value, column = $column");
1838
2
2
22
7
                                        foreach my $row(@{$data}) {
1839                                                # exists() guards: fixate() locks row hashes recursively
1840
2
16
                                                next unless exists($row->{$key}) && defined($row->{$key}) && $row->{$key} eq $value;
1841
2
3
                                                my $rc = exists($row->{$column}) ? $row->{$column} : undef;
1842
2
9
                                                $self->_trace(__LINE__, ": AUTOLOAD $key: return ", defined($rc) ? "'$rc'" : 'undef', ' from slurped data');
1843
2
26
                                                return $rc;
1844                                        }
1845
0
0
                                        $self->_debug('not found in slurped data');
1846                                }
1847                        } elsif(((scalar keys %params) == 1) && defined(my $key = $params{'entry'})) {
1848                                # Look up a single entry by its key.
1849                                # Use exists() before accessing — fixate() locks the outer hash and
1850                                # dereferencing a missing key on a locked hash throws an exception.
1851
30
24
                                my $rc;
1852
30
66
                                if(exists($data->{$key}) && defined(my $hash = $data->{$key})) {
1853
21
37
                                        if(!exists($hash->{$column})) {
1854
3
27
                                                Carp::croak(__PACKAGE__, ": There is no column $column in $table");
1855                                        }
1856
18
20
                                        $rc = $hash->{$column};
1857                                }
1858
27
36
                                if(defined($rc)) {
1859
17
45
                                        $self->_trace(__LINE__, ": AUTOLOAD $key: return '$rc' from slurped data");
1860                                } else {
1861
10
26
                                        $self->_trace(__LINE__, ": AUTOLOAD $key: return undef from slurped data");
1862                                }
1863
27
631
                                return $rc
1864                        } elsif((scalar keys %params) == 0) {
1865
7
9
                                if(wantarray) {
1866
7
10
                                        if($distinct) {
1867
7
22
27
27
7
7
29
21
36
8
                                                my %h = map { $_ => 1 } grep { defined } map { exists($_->{$column}) ? $_->{$column} : undef } values %{$data};
1868
7
19
                                                return keys %h;
1869                                        }
1870                                        # DEAD CODE: unreachable because the outer `if(wantarray && !$distinct)`
1871                                        # handles the wantarray+!distinct case. In this else branch, wantarray
1872                                        # implies $distinct (which returns above), so this line is never executed.
1873                                        # return map { exists($_->{$column}) ? $_->{$column} : undef } values %{$data}
1874                                }
1875                                # Scalar: return the first value found without building a full list
1876
0
0
0
0
                                foreach my $v (values %{$data}) {
1877
0
0
                                        return exists($v->{$column}) ? $v->{$column} : undef;
1878                                }
1879                        } else {
1880                                # Keyed data but filtering on a non-key column
1881
3
6
                                my ($key, $value) = %params;
1882
3
3
4
5
                                foreach my $row (values %{$data}) {
1883
6
23
                                        next unless exists($row->{$key}) && defined($row->{$key}) && $row->{$key} eq $value;
1884
3
4
                                        next unless exists($row->{$column});
1885
3
10
                                        if(my $rc = $row->{$column}) {
1886
3
8
                                                $self->_trace(__LINE__, ": AUTOLOAD $key: return '$rc' from slurped data");
1887
3
73
                                                return $rc
1888                                        }
1889                                }
1890                        }
1891                        return
1892
0
0
                }
1893                # Data has not been slurped in
1894
17
23
                my $id = $self->{'id'};
1895
17
47
                if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) {
1896
10
14
                        $query = "SELECT DISTINCT $column FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'";
1897
10
11
                        $done_where = 1;
1898                } else {
1899
7
13
                        $query = "SELECT DISTINCT $column FROM $table";
1900                }
1901        }
1902
20
33
        my @args;
1903        # Avoid `each` — it carries hidden iterator state across calls
1904
20
46
        for my $k (sort keys %params) {
1905                # Guard against SQL injection via column names — same rule as _build_where_conditions
1906
16
60
                Carp::croak(__PACKAGE__, ": unsafe column name '$k'")
1907                        unless $k =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/;
1908
14
19
                my $value = $params{$k};
1909
14
40
                $self->_debug(__PACKAGE__, ": AUTOLOAD adding key/value pair $k=>", defined($value) ? $value : 'NULL');
1910
14
619
                if(defined($value)) {
1911
12
30
                        $query .= $done_where ? " AND $k = ?" : " WHERE $k = ?";
1912
12
14
                        $done_where = 1;
1913
12
21
                        push @args, $value;
1914                } else {
1915
2
6
                        $query .= $done_where ? " AND $k IS NULL" : " WHERE $k IS NULL";
1916
2
3
                        $done_where = 1;
1917                }
1918        }
1919
18
24
        if(wantarray) {
1920
4
7
                $query .= " ORDER BY $column";
1921        } else {
1922
14
17
                $query .= ' LIMIT 1';
1923        }
1924
18
47
        if(scalar(@args) && $args[0]) {
1925
12
31
                $self->_debug("AUTOLOAD $query: ", join(', ', @args));
1926        } else {
1927
6
11
                $self->_debug("AUTOLOAD $query");
1928        }
1929
18
698
        my $cache;
1930
18
23
        my $key = ref($self) . '::';
1931
18
32
        if($cache = $self->{cache}) {
1932
0
0
                if(wantarray) {
1933
0
0
                        $key .= 'array ';
1934                }
1935
0
0
                if(defined($args[0])) {
1936
0
0
                        $key .= "fetchrow $query " . join(', ', @args);
1937                } else {
1938
0
0
                        $key .= "fetchrow $query";
1939                }
1940
0
0
                if(my $rc = $cache->get($key)) {
1941
0
0
                        $self->_debug('cache HIT');
1942
0
0
0
0
                        return wantarray ? @{$rc} : $rc;        # We stored a ref to the array
1943                }
1944
0
0
                $self->_debug('cache MISS');
1945        } else {
1946
18
27
                $self->_debug('cache not used');
1947        }
1948
18
588
        my $sth = $self->{$table}->prepare_cached($query) || croak($query);
1949
18
30120
        $sth->execute(@args) || croak($query);
1950
1951
18
27323
        if(wantarray) {
1952
4
14
4
5
248
23
                my @rc = map { $_->[0] } @{$sth->fetchall_arrayref()};
1953
4
8
                if($cache) {
1954
0
0
                        $cache->set($key, \@rc, $self->{'cache_duration'});       # Store a ref to the array
1955                }
1956
4
14
                Data::Reuse::fixate(@rc) if(!$self->{'no_fixate'});
1957
4
239
                return @rc;
1958        }
1959
14
89
        my $rc = $sth->fetchrow_array();     # Return the first match only
1960
14
249
        $sth->finish();
1961
14
80
        if($cache) {
1962                # Store the value, then return it — cache->set() return value is unreliable
1963
0
0
                $cache->set($key, $rc, $self->{'cache_duration'});
1964        }
1965
14
49
        return $rc;
1966}
1967
1968sub DESTROY
1969{
1970
189
59757
        if(defined($^V) && ($^V ge 'v5.14.0')) {
1971
189
300
                return if ${^GLOBAL_PHASE} eq 'DESTRUCT';       # >= 5.14.0 only
1972        }
1973
189
266
        my $self = shift;
1974
1975        # Clean up temporary file — deleting the File::Temp object triggers auto-unlink
1976
189
191
        delete $self->{'_temp_fh'};
1977
1978        # Clean up database handles
1979
189
385
        my $table_name = $self->{'table'} || ref($self);
1980
189
488
        $table_name =~ s/.*:://;
1981
1982
189
313
        if(my $dbh = delete $self->{$table_name}) {
1983
110
1724
                $dbh->disconnect() if $dbh->can('disconnect');
1984
110
1900
                $dbh->finish() if $dbh->can('finish');
1985        }
1986
1987        # Clean up Berkeley DB
1988
189
258
        if($self->{'berkeley'}) {
1989
6
7
                eval {
1990
6
6
2
11
                        untie %{$self->{'berkeley'}};
1991                };
1992
6
8
                delete $self->{'berkeley'};
1993        }
1994
1995        # Clear all other attributes to break potential circular references
1996
189
334
        foreach my $key (keys %$self) {
1997
1826
3614
                delete $self->{$key};
1998        }
1999}
2000
2001# Build the JOIN clause(s) from a single join hashref or arrayref of hashrefs.
2002# Each spec needs keys: table (required), on (required), type (default INNER).
2003sub _build_joins
2004{
2005
31
1925
        my ($self, $join_spec) = @_;
2006
2007
31
6
87
8
        my @specs = ref($join_spec) eq 'ARRAY' ? @{$join_spec} : ($join_spec);
2008
31
155
36
147
        my %valid_types = map { $_ => 1 } qw(INNER LEFT RIGHT FULL CROSS);
2009
31
28
        my @clauses;
2010
2011
31
27
        for my $j (@specs) {
2012
31
52
                my $type  = uc($j->{'type'}  // 'INNER');
2013
31
65
                my $jtable = $j->{'table'} or Carp::croak('join: missing "table"');
2014
28
61
                Carp::croak("join: unsafe table name '$jtable'")
2015                        unless $jtable =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/;
2016
28
59
                my $on     = $j->{'on'}    or Carp::croak('join: missing "on" condition');
2017
25
69
                Carp::croak("Invalid JOIN type: $type") unless $valid_types{$type};
2018
21
30
                push @clauses, "$type JOIN $jtable ON ($on)";
2019        }
2020
2021
21
44
        return join(' ', @clauses);
2022}
2023
2024# Return true when $params contains operator hashrefs, -or, or -and groupings
2025# that the simple slurp fast-path cannot handle.
2026sub _has_complex_criteria
2027{
2028
74
90
        my ($self, $params) = @_;
2029
74
110
        return 0 unless defined $params;
2030
73
176
        return 1 if exists $params->{'-or'} || exists $params->{'-and'};
2031
71
71
50
117
        for my $v (values %{$params}) {
2032
51
90
                return 1 if ref($v);
2033        }
2034
68
122
        return 0;
2035}
2036
2037# Build the WHERE clause body (everything after "WHERE") from a criteria hash.
2038# Handles -or / -and groupings then delegates per-column work to _build_where_conditions.
2039# Returns ($sql_fragment, \@bind_values).
2040sub _build_where
2041{
2042
198
2089
        my ($self, $params) = @_;
2043
2044
198
262
        $params //= {};
2045
198
198
165
230
        my %p = %{$params};     # work on a copy so we can delete -or/-and
2046
198
172
        my @clauses;
2047        my @args;
2048
2049
198
249
        if(my $or_list = delete $p{'-or'}) {
2050
11
9
                my (@sub_clauses, @sub_args);
2051
11
11
8
12
                for my $cond (@{$or_list}) {
2052
22
31
                        my ($s, $a) = $self->_build_where_conditions($cond);
2053
22
24
                        if($s) {
2054
22
24
                                push @sub_clauses, "($s)";
2055
22
22
15
24
                                push @sub_args, @{$a};
2056                        }
2057                }
2058
11
15
                if(@sub_clauses) {
2059
11
19
                        push @clauses, '(' . join(' OR ', @sub_clauses) . ')';
2060
11
34
                        push @args, @sub_args;
2061                }
2062        }
2063
198
236
        if(my $and_list = delete $p{'-and'}) {
2064
4
4
                my (@sub_clauses, @sub_args);
2065
4
4
4
6
                for my $cond (@{$and_list}) {
2066
7
8
                        my ($s, $a) = $self->_build_where_conditions($cond);
2067
7
7
                        if($s) {
2068
7
7
                                push @sub_clauses, "($s)";
2069
7
7
6
9
                                push @sub_args, @{$a};
2070                        }
2071                }
2072
4
5
                if(@sub_clauses) {
2073
4
5
                        push @clauses, '(' . join(' AND ', @sub_clauses) . ')';
2074
4
6
                        push @args, @sub_args;
2075                }
2076        }
2077
2078
198
303
        my ($more, $margs) = $self->_build_where_conditions(\%p);
2079
186
219
        if($more) {
2080
100
73
                push @clauses, $more;
2081
100
100
72
90
                push @args, @{$margs};
2082        }
2083
2084
186
352
        return (join(' AND ', @clauses), \@args);
2085}
2086
2087# Build a WHERE-body fragment for a flat col => val hash.
2088# Values may be plain scalars (= / LIKE / IS NULL) or operator hashrefs
2089# ({ '>' => n }, { -in => [...] }, { -between => [lo,hi] }, etc.).
2090sub _build_where_conditions
2091{
2092
241
3548
        my ($self, $params) = @_;
2093
2094
241
157
        my @clauses;
2095        my @args;
2096
2097
241
241
179
344
        for my $col (sort keys %{$params}) {
2098
161
144
                my $val = $params->{$col};
2099
2100                # Guard against SQL injection via column names; allow table.column notation for JOINs
2101
161
389
                Carp::croak("_build_where_conditions: unsafe column name '$col'")
2102                        unless $col =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/;
2103
2104
150
309
                if(ref($val) eq 'HASH') {
2105
55
55
36
64
                        for my $op (sort keys %{$val}) {
2106
57
43
                                my $operand = $val->{$op};
2107
57
195
                                if($op eq '-in' || $op eq '-not_in') {
2108
8
12
                                        my $sql_op = $op eq '-in' ? 'IN' : 'NOT IN';
2109
8
8
7
12
                                        my $ph = join(', ', ('?') x scalar(@{$operand}));
2110
8
13
                                        push @clauses, "$col $sql_op ($ph)";
2111
8
8
6
10
                                        push @args, @{$operand};
2112                                } elsif($op eq '-between') {
2113
5
8
                                        push @clauses, "$col BETWEEN ? AND ?";
2114
5
10
                                        push @args, $operand->[0], $operand->[1];
2115                                } elsif($op eq '-like') {
2116
3
6
                                        push @clauses, "$col LIKE ?";
2117
3
6
                                        push @args, $operand;
2118                                } elsif($op eq '-not_like') {
2119
3
5
                                        push @clauses, "$col NOT LIKE ?";
2120
3
6
                                        push @args, $operand;
2121                                } elsif($op eq '!=') {
2122
4
5
                                        if(!defined($operand)) {
2123
1
2
                                                push @clauses, "$col IS NOT NULL";
2124                                        } else {
2125
3
5
                                                push @clauses, "$col != ?";
2126
3
4
                                                push @args, $operand;
2127                                        }
2128                                } elsif($op =~ /^(?:>|<|>=|<=)$/) {
2129
33
39
                                        push @clauses, "$col $op ?";
2130
33
45
                                        push @args, $operand;
2131                                } else {
2132
1
8
                                        Carp::croak("Unknown operator '$op' for column '$col'");
2133                                }
2134                        }
2135                } elsif(ref($val)) {
2136
3
20
                        Carp::croak("$col: expected scalar or operator hashref, got ", ref($val));
2137                } elsif(!defined($val)) {
2138
5
10
                        push @clauses, "$col IS NULL";
2139                } elsif($val =~ /[%_]/) {
2140
8
15
                        push @clauses, "$col LIKE ?";
2141
8
11
                        push @args, $val;
2142                } else {
2143
79
92
                        push @clauses, "$col = ?";
2144
79
85
                        push @args, $val;
2145                }
2146        }
2147
2148
226
410
        return (join(' AND ', @clauses), \@args);
2149}
2150
2151# Test a single in-memory row value against a criteria value.
2152# $crit_val may be a plain scalar or an operator hashref.
2153# Returns true when the row value satisfies the criterion.
2154# Scan the entire BerkeleyDB tied hash, building rows as {entry=>$k, value=>$v},
2155# and filter by $params criteria using _match_criterion.
2156# Croaks when JOINs or -or/-and groupings are requested (unsupported for key-value stores).
2157sub _scan_berkeley
2158{
2159
3
3
        my ($self, $params) = @_;
2160
3
5
        $params //= {};
2161
2162
3
6
        if(delete $params->{'join'}) {
2163
0
0
                Carp::croak(ref($self), ': BerkeleyDB does not support JOINs');
2164        }
2165
3
0
3
2
0
6
        if(grep { $_ eq '-or' || $_ eq '-and' } keys %{$params}) {
2166
0
0
                Carp::croak(ref($self), ': BerkeleyDB does not support -or/-and groupings');
2167        }
2168
2169
3
4
        my $bdb = $self->{'berkeley'};
2170
3
0
3
3
0
5
        my @rows = map { { entry => $_, value => $bdb->{$_} } } keys %{$bdb};
2171
2172
3
3
4
4
        if(my @cols = keys %{$params}) {
2173                @rows = grep {
2174
0
0
0
0
                        my $row = $_;
2175
0
0
                        my $match = 1;
2176
0
0
                        for my $col (@cols) {
2177
0
0
                                unless($self->_match_criterion($row->{$col}, $params->{$col})) {
2178
0
0
                                        $match = 0;
2179
0
0
                                        last;
2180                                }
2181                        }
2182
0
0
                        $match;
2183                } @rows;
2184        }
2185
2186
3
7
        return \@rows;
2187}
2188
2189# SQL LIKE match using dynamic programming (O(m*n), no catastrophic backtracking).
2190# % matches any sequence of chars; _ matches exactly one char.  Case-insensitive.
2191sub _like_match
2192{
2193
8
9
        my ($str, $pattern) = @_;
2194
8
13
        my @s = split //, lc($str);
2195
8
11
        my @p = split //, lc($pattern);
2196
8
5
        my $m = scalar @s;
2197
8
8
        my $n = scalar @p;
2198
2199
8
38
16
39
        my @dp = map { [ (0) x ($m + 1) ] } 0 .. $n;
2200
8
10
        $dp[0][0] = 1;
2201
2202
8
12
        for my $i (1 .. $n) {
2203
30
26
                if($p[$i - 1] eq '%') {
2204
6
5
                        $dp[$i][0] = $dp[$i - 1][0];
2205
6
6
                        for my $j (1 .. $m) {
2206
26
39
                                $dp[$i][$j] = ($dp[$i - 1][$j] || $dp[$i][$j - 1]) ? 1 : 0;
2207                        }
2208                } else {
2209
24
22
                        for my $j (1 .. $m) {
2210
190
191
                                $dp[$i][$j] = ($dp[$i - 1][$j - 1]
2211                                        && ($p[$i - 1] eq '_' || $p[$i - 1] eq $s[$j - 1])) ? 1 : 0;
2212                        }
2213                }
2214        }
2215
8
25
        return $dp[$n][$m];
2216}
2217
2218sub _match_criterion
2219{
2220
107
512
        my ($self, $row_val, $crit_val) = @_;
2221
2222
107
109
        if(ref($crit_val) eq 'HASH') {
2223
50
50
31
49
                for my $op (keys %{$crit_val}) {
2224
50
43
                        my $operand = $crit_val->{$op};
2225
50
110
                        if($op eq '-in') {
2226
5
10
4
7
15
5
                                return 0 unless defined($row_val) && grep { $row_val eq $_ } @{$operand};
2227                        } elsif($op eq '-not_in') {
2228
4
10
4
7
14
5
                                return 0 if defined($row_val) && grep { $row_val eq $_ } @{$operand};
2229                        } elsif($op eq '-between') {
2230
8
27
                                return 0 unless defined($row_val) && $row_val >= $operand->[0] && $row_val <= $operand->[1];
2231                        } elsif($op eq '-like') {
2232
7
17
                                return 0 unless defined($row_val);
2233
6
12
                                return 0 unless _like_match($row_val, $operand);
2234                        } elsif($op eq '-not_like') {
2235
3
6
                                return 0 unless defined($row_val);
2236
2
4
                                return 0 if _like_match($row_val, $operand);
2237                        } elsif($op eq '!=') {
2238
8
9
                                if(!defined($operand)) {
2239
3
7
                                        return 0 unless defined($row_val);
2240                                } else {
2241
5
13
                                        return 0 unless defined($row_val) && $row_val ne $operand;
2242                                }
2243                        } elsif($op eq '>') {
2244
6
17
                                return 0 unless defined($row_val) && $row_val > $operand;
2245                        } elsif($op eq '<') {
2246
3
9
                                return 0 unless defined($row_val) && $row_val < $operand;
2247                        } elsif($op eq '>=') {
2248
3
8
                                return 0 unless defined($row_val) && $row_val >= $operand;
2249                        } elsif($op eq '<=') {
2250
3
8
                                return 0 unless defined($row_val) && $row_val <= $operand;
2251                        }
2252                }
2253
26
51
                return 1;
2254        }
2255
2256
57
201
        return !defined($row_val) && !defined($crit_val) ? 1
2257                : !defined($row_val) || !defined($crit_val) ? 0
2258                : $row_val eq $crit_val;
2259}
2260
2261# Determine the table and open the database
2262sub _open_table
2263{
2264
685
658
        my($self, $params) = @_;
2265
2266        # Get table name (remove package name prefix if present)
2267
685
2794
        my $table = $params->{'table'} || $self->{'table'} || ref($self);
2268
685
1364
        $table =~ s/.*:://;
2269
2270        # Open a connection if it's not already open.
2271        # BerkeleyDB never sets $self->{$table} (no DBI handle) or $self->{'data'},
2272        # so we also guard on $self->{'berkeley'} to avoid re-tying on every call.
2273
685
1318
        $self->_open() if((!$self->{$table}) && (!$self->{'data'}) && (!$self->{'berkeley'}));
2274
2275
680
677
        return $table;
2276}
2277
2278# Quote a SQL identifier using the current connection's dialect rules.
2279# Falls back to ANSI double-quoting when no connection is available.
2280sub _quote_identifier
2281{
2282
2
34
        my ($self, $name) = @_;
2283
2284
2
6
        my $table = $self->{'table'} || ref($self);
2285
2
5
        $table =~ s/.*:://;
2286
2
4
        if(my $dbh = $self->{$table}) {
2287
0
0
                return $dbh->quote_identifier($name);
2288        }
2289
2
6
        return qq{"$name"};
2290}
2291
2292# Determine whether a given file is a valid Berkeley DB file.
2293# It combines a fast preliminary check with a more thorough validation step for accuracy.
2294# It looks for the magic number at both byte 0 and byte 12
2295# TODO: Combine _db_0 and _db_12 as they are very similar routines
2296sub _is_berkeley_db {
2297
98
602
        my ($self, $file) = @_;
2298
2299        # Step 1: Check magic number
2300        # no autodie here: the file may not exist, and we want a silent false return
2301
98
104
        my $fh;
2302
28
28
28
98
98
126
26
138
99
1391
        do { no autodie qw(open); open $fh, '<', $file } or return 0;
2303
2
7
        binmode $fh;
2304
2305
2
937
        my $is_db = (($self->_is_berkeley_db_0($fh)) || ($self->_is_berkeley_db_12($fh)));
2306
2
26
        close $fh;
2307
2308
2
362
        if($is_db) {
2309                # Step 2: Attempt to open as Berkeley DB
2310
2311
0
0
                require DB_File && DB_File->import();
2312
2313
0
0
                my %bdb;
2314
0
0
                if(tie %bdb, 'DB_File', $file, O_RDONLY, 0644, $DB_File::DB_HASH) {
2315                        # untie %db;
2316
0
0
                        $self->{'berkeley'} = \%bdb;
2317
0
0
                        return 1;       # Successfully identified as a Berkeley DB file
2318                }
2319        }
2320
2
8
        return 0;
2321}
2322
2323# Determine whether a given file is a valid Berkeley DB file.
2324# It combines a fast preliminary check with a more thorough validation step for accuracy.
2325sub _is_berkeley_db_0
2326{
2327
3
229
        my ($self, $fh) = @_;
2328
2329        # Read the first 4 bytes (magic number)
2330
3
7
        read($fh, my $magic_bytes, 4) == 4 or return 0;
2331
2332        # Unpack both big-endian and little-endian values
2333
2
816
        my $magic_be = unpack('N', $magic_bytes);       # Big-endian
2334
2
4
        my $magic_le = unpack('V', $magic_bytes);       # Little-endian
2335
2336        # Known Berkeley DB magic numbers (in both endian formats)
2337
2
8
4
11
        my %known_magic = map { $_ => 1 } (
2338                0x00061561,     # Btree
2339                0x00053162,     # Hash
2340                0x00042253,     # Queue
2341                0x00052444,     # Recno
2342        );
2343
2344
2
17
        return($known_magic{$magic_be} || $known_magic{$magic_le});
2345}
2346
2347sub _is_berkeley_db_12
2348{
2349
3
297
        my ($self, $fh) = @_;
2350
3
4
        my $header;
2351
2352
3
5
        seek $fh, 12, 0 or return 0;
2353
3
488
        read($fh, $header, 4) or return 0;
2354
2355
2
58
        $header = substr(unpack('H*', $header), 0, 4);
2356
2357        # Berkeley DB magic numbers
2358
2
8
        return($header eq '6115' || $header eq '1561'); # Btree
2359}
2360
2361# Log and remember a message
2362sub _log
2363{
2364
1078
1817
        my ($self, $level, @messages) = @_;
2365
2366        # FIXME: add caller's function
2367        # if(($level eq 'warn') || ($level eq 'notice')) {
2368
1078
1078
778
2889
                push @{$self->{'messages'}}, { level => $level, message => join('', grep defined, @messages) };
2369        # }
2370
2371
1078
2046
        if(scalar(@messages) && (my $logger = $self->{'logger'})) {
2372
1078
2279
                $self->{'logger'}->$level(join('', grep defined, @messages));
2373        }
2374}
2375
2376sub _debug {
2377
825
1272
        my $self = shift;
2378
825
966
        $self->_log('debug', @_);
2379}
2380
2381sub _trace {
2382
249
837
        my $self = shift;
2383
249
408
        $self->_log('trace', @_);
2384}
2385
2386# Emit a warning message somewhere
2387sub _warn {
2388
1
281
        my $self = shift;
2389
1
2
        my $params = Params::Get::get_params('warning', \@_);
2390
2391
1
16
        $self->_log('warn', $params->{'warning'});
2392
1
163
        Carp::carp(join('', grep defined, $params->{'warning'}));
2393}
2394
2395# Die
2396sub _fatal {
2397
3
339
        my $self = shift;
2398
3
6
        my $params = Params::Get::get_params('warning', \@_);
2399
2400
3
51
        $self->_log('error', $params->{'warning'});
2401
3
739
        Carp::croak(join('', grep defined, $params->{'warning'}));
2402}
2403
2404 - 2560
=head1 AUTHOR

Nigel Horne, C<< <njh at nigelhorne.com> >>

=head1 SUPPORT

This module is provided as-is without any warranty.

Please report any bugs or feature requests to C<bug-database-abstraction at rt.cpan.org>,
or through the web interface at
L<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Database-Abstraction>.
I will be notified, and then you'll
automatically be notified of progress on your bug as I make changes.

=head1 MESSAGES

The table below lists every error that the module can croak or carp, what
triggers it, and how to resolve it.

=over 4

=item C<< I<Class>: abstract class >>

Direct instantiation of C<Database::Abstraction> was attempted.
Create a subclass and instantiate that instead.

=item C<< I<Class>: where are the files? >>

Neither C<directory> nor C<dsn> was supplied to C<new()>.

=item C<< I<Class>: I</path> is not a directory >>

The C<directory> argument exists on disk but is not a directory.

=item C<< I<Class>: cannot connect: I<$DBI::errstr> >>

DBI failed to connect to the given C<dsn>.  Check credentials and host.

=item C<< Can't find a file called 'I<name>' for the table I<T> in I<dir> >>

None of the probe extensions (C<.sql>, C<.psv>, C<.csv>, C<.db>, C<.xml>)
matched in C<directory>.

=item C<< I<Class>: prepare failed: I<$errstr> >>

C<prepare_cached()> returned false.  Usually a syntax error in an internally
built query; file a bug if you see this from a normal API call.

=item C<< _build_where_conditions: unsafe column name 'I<name>' >>

A criteria key contained characters outside C<[A-Za-z0-9_.]>.
This is a SQL-injection guard.  Use only valid SQL identifier characters.

=item C<< join: missing "table" >> / C<< join: missing "on" condition >>

A join spec hashref is incomplete.  Both C<table> and C<on> are required.

=item C<< Invalid JOIN type: I<TYPE> >>

C<type> in a join spec was not one of C<INNER LEFT RIGHT FULL CROSS>.

=item C<< I<Class>: Unknown column I<col> >> / C<< I<Class>: AUTOLOAD disabled >>

An AUTOLOAD call was made for a column that does not exist, or AUTOLOAD
was disabled with C<< auto_load => 0 >>.

=item C<< Usage: set_logger(logger => $logger) >>

C<set_logger()> was called without a C<logger> argument.

=item C<< Usage: execute(query => $query) >>

C<execute()> was called without a C<query> argument.

=item C<< XML slurp: I<...> is not yet supported >>

The XML file structure is too complex for slurp mode.
Use C<< max_slurp_size => 0 >> to force the DBI/XMLSimple SQL path.

=item C<< I<Class>: I<method> is meaningless on a NoSQL database >>

A relational method (C<selectall_arrayref>, C<count>, C<execute>, etc.)
was called on a BerkeleyDB backend, which only supports key-value lookup
via C<fetchrow_hashref>.

=back

=head1 KNOWN LIMITATIONS

=over 4

=item *

B<Read-only.>  No INSERT, UPDATE, or DELETE is provided.  C<execute()>
runs raw read-only SQL.

=item *

B<Default CSV separator is C<!>>, not C<,>, for historical reasons.
Pass C<< sep_char => ',' >> for standard RFC 4180 files.

=item *

B<Primary-key column is named C<entry>>, not C<key>, because C<key>
is a SQL reserved word.  Override with the C<id> parameter.

=item *

B<XML slurp is limited.>  Only simple flat XML structures are supported
in slurp mode.  Multi-key or deeply nested documents will croak.
Force SQL mode with C<< max_slurp_size => 0 >> if slurp fails.

=item *

B<Unique key assumption in slurp mode.>  Duplicate values in the key
column silently overwrite earlier rows.  Disable slurp with
C<< max_slurp_size => 0 >> if duplicates are expected.

=item *

B<BerkeleyDB does not support joins or the chained query builder.>

=item *

B<Column names must be valid SQL identifiers> (letters, digits,
underscores, and a single dot for C<table.column> join notation).
Other characters will cause a croak.

=item *

B<count() cache is opportunistic.>  Count results are served from cache
only when a prior C<selectall_arrayref()> or C<count()> call with the
same criteria has already populated it.

=back

=head1 SEE ALSO

=over 4

=item * L<Database::Abstraction::Query> - chained query builder

=item * L<Configure an Object at Runtime|Object::Configure>

=item * L<Test Dashboard|https://nigelhorne.github.io/Database-Abstraction/coverage/>

=back

=head1 LICENSE AND COPYRIGHT

Copyright 2015-2026 Nigel Horne.

Usage is subject to the GPL2 licence terms.
If you use it,
please let me know.

=cut
2561
25621;