| File: | blib/lib/Database/Abstraction.pm |
| Coverage: | 80.2% |
| line | stmt | bran | cond | sub | time | code |
|---|---|---|---|---|---|---|
| 1 | package Database::Abstraction; | |||||
| 2 | ||||||
| 3 | # Author Nigel Horne: njh@nigelhorne.com | |||||
| 4 | # Copyright (C) 2015-2026, Nigel Horne | |||||
| 5 | ||||||
| 6 | # Usage is subject to licence terms. | |||||
| 7 | # The licence terms of this software are as follows: | |||||
| 8 | # Personal single user, single computer use: GPL2 | |||||
| 9 | # All other users (for example, Commercial, Charity, Educational, Government) | |||||
| 10 | # must apply in writing for a licence for use from Nigel Horne at the | |||||
| 11 | # above e-mail. | |||||
| 12 | ||||||
| 13 | # TODO: Switch "entry" to off by default, and enable by passing 'entry' | |||||
| 14 | # though that wouldn't be so nice for AUTOLOAD | |||||
| 15 | # TODO: support a directory hierarchy of databases | |||||
| 16 | # TODO: consider returning an object or array of objects, rather than hashes | |||||
| 17 | # TODO: Add redis database - could be of use for Geo::Coder::Free | |||||
| 18 | # use select() to select a database - use the table arg | |||||
| 19 | # new(database => 'redis://servername'); | |||||
| 20 | # TODO: Add a "key" property, defaulting to "entry", which would be the name of the key | |||||
| 21 | # TODO: The maximum number to return should be tuneable (as a LIMIT) | |||||
| 22 | # TODO: Add full CRUD support | |||||
| 23 | # TODO: It would be better for the default sep_char to be ',' rather than '!' | |||||
| 24 | # TODO: Other databases e.g., Redis, noSQL, remote databases such as MySQL, PostgreSQL | |||||
| 25 | # TODO: The no_entry/entry terminology is confusing. Replace with no_id/id_column | |||||
| 26 | # TODO: Add support for DBM::Deep | |||||
| 27 | # TODO: Log queries and the time that they took to execute per database | |||||
| 28 | ||||||
| 29 | 28 28 28 | 2800543 24 603 | use warnings; | |||
| 30 | 28 28 28 | 51 21 275 | use strict; | |||
| 31 | 28 28 28 | 4348 146257 59 | use autodie qw(:all); | |||
| 32 | ||||||
| 33 | 28 28 28 | 193957 12522 58 | use boolean; | |||
| 34 | 28 28 28 | 881 23 538 | use Carp; | |||
| 35 | 28 28 28 | 5719 524381 85 | use Class::Abstract; | |||
| 36 | 28 28 28 | 9229 41013 631 | use Data::Reuse; | |||
| 37 | 28 28 28 | 20433 212814 920 | use DBI; | |||
| 38 | 28 28 28 | 89 22 3050 | use Fcntl; # For O_RDONLY | |||
| 39 | 28 28 28 | 63 22 720 | use Cwd; | |||
| 40 | 28 28 28 | 53 23 323 | use File::Spec; | |||
| 41 | 28 28 28 | 6742 120709 970 | use File::Temp; | |||
| 42 | 28 28 28 | 65 23 830 | use List::Util qw(all); | |||
| 43 | 28 28 28 | 8358 950598 665 | use Log::Abstraction 0.26; | |||
| 44 | 28 28 28 | 7902 142659 555 | use Object::Configure 0.16; | |||
| 45 | 28 28 28 | 106 151 503 | use Params::Get 0.13; | |||
| 46 | 28 28 28 | 56 21 515 | use Return::Set qw(set_return); | |||
| 47 | 28 28 28 | 44 25 557 | use Scalar::Util; | |||
| 48 | ||||||
| 49 | our %defaults; | |||||
| 50 | 28 28 28 | 47 24 153893 | use constant DEFAULT_MAX_SLURP_SIZE => 16 * 1024; # CSV files <= than this size are read into memory | |||
| 51 | ||||||
| 52 - 60 | =head1 NAME Database::Abstraction - Read-only Database Abstraction Layer (ORM) =head1 VERSION Version 0.36 =cut | |||||
| 61 | ||||||
| 62 | our $VERSION = '0.36'; | |||||
| 63 | ||||||
| 64 - 338 | =head1 DESCRIPTION
C<Database::Abstraction> is a read-only ORM for Perl that gives a uniform
interface over CSV, PSV, XML, SQLite, and BerkeleyDB files - without writing
any SQL.
Key features:
=over 4
=item *
B<No SQL required.> Use plain Perl method calls for simple lookups and
scans; switch storage formats without changing application code.
=item *
B<Rich query criteria.> Pass plain values, SQL wildcards, C<undef> (IS NULL),
comparison operators (C<< > >> C<< < >> C<< >= >> C<< <= >> C<!=>), pattern
operators (C<-like>, C<-not_like>), set operators (C<-in>, C<-not_in>,
C<-between>), and logical groupings (C<-or>, C<-and>).
=item *
B<Automatic joins.> Add a C<join> parameter to any select method to
combine tables with INNER, LEFT, RIGHT, FULL, or CROSS joins.
=item *
B<Chained query builder.> The C<query()> method returns a
L<Database::Abstraction::Query> object for fluent, composable queries:
C<< $db->query->where(...)->order_by(...)->limit(...)->all() >>.
=item *
B<Schema introspection.> C<columns()> lists column names; C<schema()>
returns full type/nullability metadata, using native driver introspection
(C<PRAGMA table_info> for SQLite, C<column_info> for others).
=item *
B<DSN portability.> Pass a C<dsn> (plus optional C<username>/C<password>)
to connect to any DBI-supported database (SQLite, PostgreSQL, MySQL, ...)
instead of pointing at a local file.
=item *
B<Performance.> Small files are slurped into a RAM hash for sub-millisecond
lookups. All DBI statement handles are cached with C<prepare_cached()>.
A CHI-compatible cache layer is also supported.
=back
=head1 SYNOPSIS
# 1. Create a thin subclass for your table (e.g. Database/Foo.pm)
package Database::Foo;
use parent 'Database::Abstraction';
# 2. Open the database - file is auto-detected from the class name
# (looks for foo.sql / foo.psv / foo.csv / foo.xml / foo.db)
my $db = Database::Foo->new(directory => '/path/to/data');
# 3. Simple lookups -----------------------------------------------
# Fetch one row
my $row = $db->fetchrow_hashref(entry => 'key1');
# Fetch all rows matching a criterion
my $rows = $db->selectall_arrayref(status => 'active');
# Column shortcut via AUTOLOAD
my $name = $db->name(entry => 'key1');
# 4. Rich criteria ------------------------------------------------
# Comparison operators
my $high = $db->selectall_arrayref(score => { '>' => 90 });
# Set membership
my $selected = $db->selectall_arrayref(
name => { -in => ['Alice', 'Bob'] }
);
# Range
my $mid = $db->selectall_arrayref(
score => { -between => [60, 80] }
);
# OR grouping
my $either = $db->selectall_arrayref(
-or => [
{ status => 'active' },
{ score => { '>' => 95 } },
]
);
# 5. Joins --------------------------------------------------------
my $joined = $db->selectall_arrayref(
join => { table => 'dept', on => 'foo.dept_id = dept.id', type => 'LEFT' }
);
# 6. Chained query builder ----------------------------------------
my $results = $db->query
->where(status => 'active')
->where(score => { '>=' => 80 })
->order_by('score DESC')
->limit(10)
->all();
my $first = $db->query->where(name => 'Alice')->first();
my $count = $db->query->where(status => 'active')->count();
# 7. Connect via DSN (PostgreSQL, MySQL, SQLite, ...) ---------------
my $db2 = Database::Foo->new(
dsn => 'dbi:Pg:dbname=mydb;host=db.example.com',
username => 'myuser',
password => 's3cret',
);
# 8. Schema introspection -----------------------------------------
my $cols = $db->columns(); # ['entry', 'name', 'score', ...]
my $schema = $db->schema(); # { name => { type=>'TEXT', nullable=>1, ... }, ... }
=head1 QUICK START EXAMPLE
If F</var/dat/foo.csv> contains:
"customer_id","name"
"plugh","John"
"xyzzy","Jane"
Create a driver in F<.../Database/foo.pm>:
package Database::foo;
use parent 'Database::Abstraction';
# Regular CSV: no entry column, comma-separated
sub new {
my ($class, %args) = @_;
return $class->SUPER::new(no_entry => 1, sep_char => ',', %args);
}
Then query it:
my $foo = Database::foo->new(directory => '/var/dat');
# Prints "John"
print 'Customer: ', $foo->name(customer_id => 'plugh'), "\n";
# Returns { customer_id => 'xyzzy', name => 'Jane' }
my $row = $foo->fetchrow_hashref(customer_id => 'xyzzy');
=head1 FILE FORMATS
The module probes the C<directory> for files in this priority order:
=over 4
=item 1. C<SQLite>
File ending C<.sql>
=item 2. C<PSV>
Pipe-separated file, ending C<.psv>
=item 3. C<CSV>
Comma (or custom) separated file, ending C<.csv> or C<.db>; can be
gzipped. B<Note:> the default separator is C<!> not C<,> for historical
reasons - pass C<< sep_char => ',' >> for standard CSVs.
=item 4. C<XML>
File ending C<.xml>
=item 5. C<BerkeleyDB>
Binary key-value file ending C<.db>
=back
Pass C<dsn> to bypass file detection entirely and connect via any DBI driver.
=head1 QUERY CRITERIA
All select methods (C<selectall_arrayref>, C<selectall_array>,
C<fetchrow_hashref>, C<count>) accept the same criteria syntax.
=head2 Plain value
status => 'active' # status = 'active'
name => undef # name IS NULL
Values containing C<%> or C<_> are matched with C<LIKE>:
name => 'A%' # name LIKE 'A%'
=head2 Comparison operator hashref
score => { '>' => 90 } # score > 90
score => { '<' => 50 } # score < 50
score => { '>=' => 80 } # score >= 80
score => { '<=' => 100 } # score <= 100
score => { '!=' => 0 } # score != 0
Multiple operators on one column are ANDed:
score => { '>' => 60, '<' => 90 } # 60 < score < 90
=head2 Pattern matching
name => { -like => 'A%' } # name LIKE 'A%'
name => { -not_like => 'Z%' } # name NOT LIKE 'Z%'
=head2 Set membership
name => { -in => ['Alice', 'Bob'] } # name IN (...)
name => { -not_in => ['Alice', 'Bob'] } # name NOT IN (...)
=head2 Range
score => { -between => [60, 90] } # score BETWEEN 60 AND 90
=head2 Logical groupings
C<-or> and C<-and> take an arrayref of condition hashrefs:
-or => [
{ status => 'active' },
{ score => { '>' => 95 } },
]
-and => [
{ status => 'active' },
{ score => { '>=' => 80 } },
]
=head2 Joins
Any select method accepts a C<join> key with a hashref (or arrayref of
hashrefs) describing the join:
join => {
table => 'dept',
on => 'employees.dept_id = dept.id',
type => 'LEFT', # INNER (default) | LEFT | RIGHT | FULL | CROSS
}
# Multiple joins
join => [
{ table => 'dept', on => 'e.dept_id = dept.id' },
{ table => 'country', on => 'e.country_id = country.id' },
]
=head1 SUBROUTINES/METHODS
=head2 init
Set class-level defaults shared by all instances.
Database::Abstraction::init(directory => '../data');
Accepts the same parameters as L</new>. Returns a reference to the
current defaults hash, so you can read them back:
my $defaults = Database::Abstraction::init();
print $defaults->{'directory'}, "\n";
=cut | |||||
| 339 | ||||||
| 340 | # Subroutine to initialize with args | |||||
| 341 | sub init | |||||
| 342 | { | |||||
| 343 | 111 | 764454 | if(my $params = Params::Get::get_params(undef, @_)) { | |||
| 344 | 105 | 1401 | if(($params->{'expires_in'} && !$params->{'cache_duration'})) { | |||
| 345 | # Compatibility with CHI | |||||
| 346 | 4 | 5 | $params->{'cache_duration'} = $params->{'expires_in'}; | |||
| 347 | } | |||||
| 348 | ||||||
| 349 | 105 105 | 747 248 | %defaults = (%defaults, %{$params}); | |||
| 350 | 105 | 382 | $defaults{'cache_duration'} ||= '1 hour'; | |||
| 351 | } | |||||
| 352 | ||||||
| 353 | 111 | 6974 | return \%defaults | |||
| 354 | } | |||||
| 355 | ||||||
| 356 - 366 | =head2 import
The module can be initialised by the C<use> directive.
use Database::Abstraction 'directory' => '/etc/data';
or
use Database::Abstraction { 'directory' => '/etc/data' };
=cut | |||||
| 367 | ||||||
| 368 | sub import | |||||
| 369 | { | |||||
| 370 | 88 | 650040 | my $pkg = shift; | |||
| 371 | ||||||
| 372 | 88 | 200 | if((scalar(@_) % 2) == 0) { | |||
| 373 | 86 | 101 | my %h = @_; | |||
| 374 | 86 | 181 | init(Object::Configure::configure($pkg, \%h)); | |||
| 375 | } elsif((scalar(@_) == 1) && (ref($_[0]) eq 'HASH')) { | |||||
| 376 | 2 | 6 | init(Object::Configure::configure($pkg, $_[0])); | |||
| 377 | } elsif(scalar(@_) > 0) { # >= 3 would also work here | |||||
| 378 | 0 | 0 | init(\@_); | |||
| 379 | } | |||||
| 380 | } | |||||
| 381 | ||||||
| 382 - 510 | =head2 new Create an object pointing to a read-only database. Accepts arguments as a hash, a hashref, or - as a shortcut - a single bare string which is taken to be C<directory>. =head3 Connection parameters =over 4 =item * C<directory> Directory containing the data files. The module probes this directory for files named after the subclass (see L</FILE FORMATS>). Required unless C<dsn> is given. =item * C<dsn> A DBI data-source string (e.g. C<dbi:SQLite:dbname=/path/to/db> or C<dbi:Pg:dbname=mydb;host=db.example.com>). When present, file detection is skipped entirely and the DSN is used directly. The SQL dialect is inferred from the DSN prefix (C<sqlite>, C<postgres>, C<mysql>). =item * C<username> Database username. Used only with C<dsn>; ignored for file-based backends. =item * C<password> Database password. Used only with C<dsn>; ignored for file-based backends. =item * C<dbname> Override the filename stem searched in C<directory> (default: the table name derived from the class name). =item * C<filename> Override the full filename (relative to C<directory>). Takes precedence over C<dbname>. =back =head3 Behaviour parameters =over 4 =item * C<no_entry> Set to C<1> when the table has no key column (standard CSVs, for example). Default is C<0> (keyed on C<entry>). =item * C<id> Name of the key column. Default is C<entry>. =item * C<sep_char> Field separator for CSV/PSV files. Default is C<!> - pass C<< sep_char => ',' >> for standard comma-separated files. =item * C<max_slurp_size> Files smaller than this (in bytes) are loaded entirely into memory for fast lookups. Default is 16 KB. Set to C<0> to force SQL mode for all sizes. =item * C<no_fixate> Set to C<1> to return mutable arrays. Default is C<0> (arrays are made read-only via L<Data::Reuse>). =item * C<auto_load> Set to C<0> to disable the AUTOLOAD column shortcut. Default is C<1> (enabled). =back =head3 Caching and logging =over 4 =item * C<cache> A L<CHI>-compatible cache object. When set, query results are stored and retrieved from the cache. =item * C<cache_duration> / C<expires_in> TTL for cached results. Default is C<'1 hour'>. C<expires_in> is a synonym for compatibility with L<CHI>. =item * C<logger> An object that understands C<warn()> and C<trace()> (e.g. L<Log::Log4perl>, L<Log::Any>), a code reference, or a filename. =item * C<config_file> Path to a YAML, XML, or INI configuration file whose keys are merged into the constructor arguments. Loaded via L<Object::Configure>. =back =head3 Notes =over 4 =item * If no arguments are set, class-level defaults set via C<init()> or C<use> are used. =item * Slurp mode assumes the key column (C<entry>) is unique. If it is not, searches will be incomplete - disable slurp mode by setting C<< max_slurp_size => 0 >>. =item * Passing an existing object as C<$class> clones it, merging any new arguments. =back =cut | |||||
| 511 | ||||||
| 512 | sub new { | |||||
| 513 | 210 | 1310896 | my $class = shift; | |||
| 514 | 210 | 197 | my %args; | |||
| 515 | ||||||
| 516 | 210 | 471 | Class::Abstract::check_abstract($class); # enforces abstract contract | |||
| 517 | ||||||
| 518 | # Handle hash or hashref arguments | |||||
| 519 | 210 | 1790 | if((scalar(@_) == 1) && !ref($_[0])) { | |||
| 520 | 56 | 202 | $args{'directory'} = $_[0]; | |||
| 521 | } elsif(my $params = Params::Get::get_params(undef, @_)) { | |||||
| 522 | 144 144 | 1986 261 | %args = %{$params}; | |||
| 523 | } | |||||
| 524 | ||||||
| 525 | 210 | 1271 | if(!defined($class)) { | |||
| 526 | 0 | 0 | if((scalar keys %args) > 0) { | |||
| 527 | # Using Database::Abstraction->new(), not Database::Abstraction::new() | |||||
| 528 | 0 | 0 | carp(__PACKAGE__, ' use ->new() not ::new() to instantiate'); | |||
| 529 | 0 | 0 | return; | |||
| 530 | } | |||||
| 531 | # FIXME: this only works when no arguments are given | |||||
| 532 | 0 | 0 | $class = __PACKAGE__; | |||
| 533 | } elsif($class eq __PACKAGE__) { | |||||
| 534 | 6 | 57 | croak("$class: abstract class"); | |||
| 535 | } elsif(Scalar::Util::blessed($class)) { | |||||
| 536 | # If $class is an object, clone it with new arguments | |||||
| 537 | 6 6 | 6 24 | return bless { %{$class}, %args }, ref($class); | |||
| 538 | } | |||||
| 539 | ||||||
| 540 | # Load the configuration from a config file, if provided | |||||
| 541 | 198 198 | 187 375 | %args = %{Object::Configure::configure($class, \%args)}; | |||
| 542 | ||||||
| 543 | # Normalise logger: wrap code-refs, filenames, and strings in Log::Abstraction | |||||
| 544 | # so that the rest of the code can always call ->$level(...) uniformly. | |||||
| 545 | 198 | 384504 | if(defined $args{'logger'} && !Scalar::Util::blessed($args{'logger'})) { | |||
| 546 | 0 | 0 | $args{'logger'} = Log::Abstraction->new($args{'logger'}); | |||
| 547 | } | |||||
| 548 | ||||||
| 549 | 198 | 527 | unless($args{'dsn'} || $defaults{'dsn'}) { | |||
| 550 | 182 | 385 | croak("$class: where are the files?") unless($args{'directory'} || $defaults{'directory'}); | |||
| 551 | ||||||
| 552 | 177 | 1258 | croak("$class: ", $args{'directory'} || $defaults{'directory'}, ' is not a directory') unless(-d ($args{'directory'} || $defaults{'directory'})); | |||
| 553 | } | |||||
| 554 | ||||||
| 555 | # Validate the primary-key column name to prevent SQL injection via ORDER BY / WHERE | |||||
| 556 | 187 | 276 | for my $src (\%defaults, \%args) { | |||
| 557 | 374 | 1017 | if(defined $src->{'id'}) { | |||
| 558 | croak("$class: unsafe id column name '$src->{id}'") | |||||
| 559 | 14 | 60 | unless $src->{'id'} =~ /^[a-zA-Z_][a-zA-Z0-9_]*$/; | |||
| 560 | } | |||||
| 561 | } | |||||
| 562 | ||||||
| 563 | # Defaults are set first so that %args keys override them | |||||
| 564 | 187 | 906 | return bless { | |||
| 565 | no_entry => 0, | |||||
| 566 | no_fixate => 0, | |||||
| 567 | id => 'entry', | |||||
| 568 | cache_duration => '1 hour', | |||||
| 569 | max_slurp_size => DEFAULT_MAX_SLURP_SIZE, | |||||
| 570 | %defaults, | |||||
| 571 | %args, | |||||
| 572 | }, $class; | |||||
| 573 | } | |||||
| 574 | ||||||
| 575 - 579 | =head2 set_logger Sets the class, code reference, or file that will be used for logging. =cut | |||||
| 580 | ||||||
| 581 | sub set_logger | |||||
| 582 | { | |||||
| 583 | 16 | 4040 | my $self = shift; | |||
| 584 | 16 | 29 | my $params = Params::Get::get_params('logger', @_); | |||
| 585 | ||||||
| 586 | 13 | 192 | if(my $logger = $params->{'logger'}) { | |||
| 587 | 12 | 23 | if(Scalar::Util::blessed($logger)) { | |||
| 588 | 4 | 14 | $self->{'logger'} = $logger; | |||
| 589 | } else { | |||||
| 590 | 8 | 16 | $self->{'logger'} = Log::Abstraction->new($logger); | |||
| 591 | } | |||||
| 592 | 12 | 143 | return $self; | |||
| 593 | } | |||||
| 594 | 1 | 16 | Carp::croak('Usage: set_logger(logger => $logger)') | |||
| 595 | } | |||||
| 596 | ||||||
| 597 | # Open the database connection based on the specified type (e.g., SQLite, CSV). | |||||
| 598 | # Read the data into memory or establish a connection to the database file. | |||||
| 599 | # column_names allows the column names to be overridden on CSV files | |||||
| 600 | ||||||
| 601 | sub _open | |||||
| 602 | { | |||||
| 603 | # Enforce that _open is only reachable from within this class hierarchy; | |||||
| 604 | # caller() returns the calling package name as a plain string. | |||||
| 605 | 123 123 123 | 346 241 1417 | do { my $c = (caller)[0]; Carp::croak('Illegal Operation: _open may only be called within ', __PACKAGE__) unless $c && $c->isa(__PACKAGE__) }; | |||
| 606 | ||||||
| 607 | 122 | 116 | my $self = shift; | |||
| 608 | 122 | 208 | my $params = Params::Get::get_params(undef, @_); | |||
| 609 | ||||||
| 610 | 122 | 1910 | $params->{'sep_char'} ||= $self->{'sep_char'} ? $self->{'sep_char'} : '!'; | |||
| 611 | 122 | 253 | my $max_slurp_size = $params->{'max_slurp_size'} || $self->{'max_slurp_size'}; | |||
| 612 | ||||||
| 613 | 122 | 222 | my $table = $self->{'table'} || ref($self); | |||
| 614 | 122 | 221 | $table =~ s/.*:://; | |||
| 615 | ||||||
| 616 | 122 | 344 | $self->_trace(ref($self), ": _open $table"); | |||
| 617 | ||||||
| 618 | 122 | 4293 | return if($self->{$table}); | |||
| 619 | ||||||
| 620 | # Read in the database | |||||
| 621 | 122 | 108 | my $dbh; | |||
| 622 | ||||||
| 623 | # DSN-based connection bypasses file detection entirely | |||||
| 624 | 122 | 357 | if(my $dsn = $self->{'dsn'} || $defaults{'dsn'}) { | |||
| 625 | 14 | 78 | require DBI && DBI->import() unless DBI->can('connect'); | |||
| 626 | ||||||
| 627 | 14 | 22 | my $dialect = 'generic'; | |||
| 628 | 14 14 | 41 22 | if ($dsn =~ /^dbi:SQLite:/i) { $dialect = 'sqlite' } | |||
| 629 | 0 | 0 | elsif ($dsn =~ /^dbi:Pg:/i) { $dialect = 'postgres' } | |||
| 630 | 0 | 0 | elsif ($dsn =~ /^dbi:mysql:/i) { $dialect = 'mysql' } | |||
| 631 | 14 | 20 | $self->{'_dialect'} = $dialect; | |||
| 632 | ||||||
| 633 | $dbh = DBI->connect( | |||||
| 634 | $dsn, | |||||
| 635 | $self->{'username'}, | |||||
| 636 | 14 | 73 | $self->{'password'}, | |||
| 637 | { RaiseError => 1, AutoCommit => 1 }, | |||||
| 638 | ) or Carp::croak(ref($self), ": cannot connect: $DBI::errstr"); | |||||
| 639 | ||||||
| 640 | 14 | 4060 | if($dialect eq 'sqlite') { | |||
| 641 | 14 | 40 | $dbh->do('PRAGMA synchronous = OFF'); | |||
| 642 | 14 | 215 | $dbh->do('PRAGMA cache_size = -4096'); | |||
| 643 | 14 | 111 | $dbh->do('PRAGMA journal_mode = OFF'); | |||
| 644 | 14 | 108 | $dbh->do('PRAGMA temp_store = MEMORY'); | |||
| 645 | 14 | 90 | $dbh->do('PRAGMA mmap_size = 1048576'); | |||
| 646 | 14 | 122 | $dbh->sqlite_busy_timeout(100000); | |||
| 647 | } | |||||
| 648 | ||||||
| 649 | 14 | 55 | $self->{'type'} = 'DBI'; | |||
| 650 | 14 | 32 | $self->{$table} = $dbh; | |||
| 651 | 14 | 29 | $self->{'_updated'} = time(); | |||
| 652 | 14 | 26 | return $self; | |||
| 653 | } | |||||
| 654 | ||||||
| 655 | 108 | 1802 | my $dir = Cwd::abs_path($self->{'directory'} || $defaults{'directory'}); | |||
| 656 | 108 | 245 | my $dbname = $self->{'dbname'} || $defaults{'dbname'} || $table; | |||
| 657 | 108 | 402 | Carp::croak(ref($self), ": unsafe dbname '$dbname'") | |||
| 658 | unless $dbname =~ /^[a-zA-Z0-9_.-]+$/ && $dbname !~ /\.\./; | |||||
| 659 | 108 | 531 | my $slurp_file = File::Spec->catfile($dir, "$dbname.sql"); | |||
| 660 | ||||||
| 661 | 108 | 281 | $self->_debug("_open: try to open $slurp_file"); | |||
| 662 | ||||||
| 663 | # Look at various places to find the file and derive the file type from the file's name | |||||
| 664 | 108 | 2826 | if(-r $slurp_file) { | |||
| 665 | # SQLite file | |||||
| 666 | 12 | 44 | require DBI && DBI->import() unless DBI->can('connect'); | |||
| 667 | ||||||
| 668 | 12 | 536 | require DBD::SQLite::Constants; | |||
| 669 | 12 | 1101 | $dbh = DBI->connect("dbi:SQLite:dbname=$slurp_file", undef, undef, { | |||
| 670 | sqlite_open_flags => DBD::SQLite::Constants::SQLITE_OPEN_READONLY(), | |||||
| 671 | }); | |||||
| 672 | } | |||||
| 673 | 108 | 3425 | if($dbh) { | |||
| 674 | 12 | 29 | $dbh->do('PRAGMA synchronous = OFF'); | |||
| 675 | 12 | 139 | $dbh->do('PRAGMA cache_size = -4096'); # Use 4MB cache - negative = KB) | |||
| 676 | 12 | 91 | $dbh->do('PRAGMA journal_mode = OFF'); # Read-only, no journal needed | |||
| 677 | 12 | 84 | $dbh->do('PRAGMA temp_store = MEMORY'); # Store temp data in RAM | |||
| 678 | 12 | 69 | $dbh->do('PRAGMA mmap_size = 1048576'); # Use 1MB memory-mapped I/O | |||
| 679 | 12 | 89 | $dbh->sqlite_busy_timeout(100000); # 10s | |||
| 680 | 12 | 30 | $self->_debug("read in $table from SQLite $slurp_file"); | |||
| 681 | 12 | 259 | $self->{'type'} = 'DBI'; | |||
| 682 | } elsif($self->_is_berkeley_db(File::Spec->catfile($dir, "$dbname.db"))) { | |||||
| 683 | 0 | 0 | $self->_debug("$table is a BerkeleyDB file"); | |||
| 684 | 0 | 0 | $self->{'type'} = 'BerkeleyDB'; | |||
| 685 | } else { | |||||
| 686 | 96 | 109 | my $fin; | |||
| 687 | # File::pfopen splits $path on ':' which breaks Windows drive letters | |||||
| 688 | # (C:\foo becomes ['C', '\foo']). Since we always have a single directory | |||||
| 689 | # we use File::Spec->catfile directly â same behaviour, portable. | |||||
| 690 | 96 | 145 | for my $ext (qw(csv.gz db.gz)) { | |||
| 691 | 192 | 662 | my $candidate = File::Spec->catfile($dir, "$dbname.$ext"); | |||
| 692 | 192 | 1296 | next unless -r $candidate; | |||
| 693 | 0 | 0 | open($fin, '<', $candidate) or next; | |||
| 694 | 0 | 0 | $slurp_file = $candidate; | |||
| 695 | 0 | 0 | last; | |||
| 696 | } | |||||
| 697 | 96 | 387 | if(defined($slurp_file) && (-r $slurp_file)) { | |||
| 698 | 0 | 0 | require Gzip::Faster; | |||
| 699 | 0 | 0 | Gzip::Faster->import(); | |||
| 700 | ||||||
| 701 | 0 | 0 | close($fin); | |||
| 702 | 0 | 0 | $fin = File::Temp->new(SUFFIX => '.csv', UNLINK => 1); | |||
| 703 | 0 | 0 | print $fin gunzip_file($slurp_file); | |||
| 704 | 0 | 0 | $fin->flush(); | |||
| 705 | 0 | 0 | $slurp_file = $fin->filename(); | |||
| 706 | 0 | 0 | $self->{'_temp_fh'} = $fin; # Keep object alive; auto-unlinks at DESTROY | |||
| 707 | } else { | |||||
| 708 | 96 | 307 | my $psv = File::Spec->catfile($dir, "$dbname.psv"); | |||
| 709 | 96 | 416 | if(-r $psv && open($fin, '<', $psv)) { | |||
| 710 | # Pipe separated file | |||||
| 711 | 8 | 1982 | $slurp_file = $psv; | |||
| 712 | 8 | 12 | $params->{'sep_char'} = '|'; | |||
| 713 | } else { | |||||
| 714 | # CSV or BerkeleyDB-extension file | |||||
| 715 | 88 | 96 | for my $ext (qw(csv db)) { | |||
| 716 | 102 | 280 | my $candidate = File::Spec->catfile($dir, "$dbname.$ext"); | |||
| 717 | 102 | 463 | next unless -r $candidate; | |||
| 718 | 75 | 179 | open($fin, '<', $candidate) or next; | |||
| 719 | 75 | 15987 | $slurp_file = $candidate; | |||
| 720 | 75 | 88 | last; | |||
| 721 | } | |||||
| 722 | } | |||||
| 723 | } | |||||
| 724 | 96 | 280 | if(my $filename = $self->{'filename'} || $defaults{'filename'}) { | |||
| 725 | 1 | 3 | Carp::croak(ref($self), ": unsafe filename '$filename'") | |||
| 726 | unless $filename =~ /^[a-zA-Z0-9_.-]+$/ && $filename !~ /\.\./; | |||||
| 727 | 1 | 2 | $self->_debug("Looking for $filename in $dir"); | |||
| 728 | 1 | 17 | $slurp_file = File::Spec->catfile($dir, $filename); | |||
| 729 | } | |||||
| 730 | 96 | 532 | if(defined($slurp_file) && (-r $slurp_file)) { | |||
| 731 | 84 | 202 | close($fin) if(defined($fin)); | |||
| 732 | 84 | 7594 | my $sep_char = $params->{'sep_char'}; | |||
| 733 | ||||||
| 734 | 84 | 216 | $self->_debug(__LINE__, ' of ', __PACKAGE__, ": slurp_file = $slurp_file, sep_char = $sep_char"); | |||
| 735 | ||||||
| 736 | 84 | 2059 | if($params->{'column_names'}) { | |||
| 737 | $dbh = DBI->connect("dbi:CSV:db_name=$slurp_file", undef, undef, | |||||
| 738 | { | |||||
| 739 | csv_sep_char => $sep_char, | |||||
| 740 | csv_tables => { | |||||
| 741 | $table => { | |||||
| 742 | 1 | 7 | col_names => $params->{'column_names'}, | |||
| 743 | }, | |||||
| 744 | }, | |||||
| 745 | f_dir => $dir, | |||||
| 746 | RaiseError => 1, | |||||
| 747 | PrintError => 0 | |||||
| 748 | } | |||||
| 749 | ); | |||||
| 750 | } else { | |||||
| 751 | 83 | 418 | $dbh = DBI->connect("dbi:CSV:db_name=$slurp_file", undef, undef, { csv_sep_char => $sep_char, f_dir => $dir, RaiseError => 1 }); | |||
| 752 | } | |||||
| 753 | 84 | 1116154 | $dbh->{'RaiseError'} = 1; | |||
| 754 | ||||||
| 755 | 84 | 514 | $self->_debug("read in $table from CSV $slurp_file"); | |||
| 756 | ||||||
| 757 | 84 | 2841 | $dbh->{csv_tables}->{$table} = { | |||
| 758 | allow_loose_quotes => 1, | |||||
| 759 | blank_is_undef => 1, | |||||
| 760 | empty_is_undef => 1, | |||||
| 761 | binary => 1, | |||||
| 762 | f_file => $slurp_file, | |||||
| 763 | escape_char => '\\', | |||||
| 764 | sep_char => $sep_char, | |||||
| 765 | # Don't do this, causes "Bizarre copy of HASH | |||||
| 766 | # in scalar assignment in error_diag | |||||
| 767 | # RT121127 | |||||
| 768 | # auto_diag => 1, | |||||
| 769 | auto_diag => 0, | |||||
| 770 | # Don't do this, it causes "Attempt to free unreferenced scalar" | |||||
| 771 | # callbacks => { | |||||
| 772 | # after_parse => sub { | |||||
| 773 | # my ($csv, @rows) = @_; | |||||
| 774 | # my @rc; | |||||
| 775 | # foreach my $row(@rows) { | |||||
| 776 | # if($row->[0] !~ /^#/) { | |||||
| 777 | # push @rc, $row; | |||||
| 778 | # } | |||||
| 779 | # } | |||||
| 780 | # return @rc; | |||||
| 781 | # } | |||||
| 782 | # } | |||||
| 783 | }; | |||||
| 784 | ||||||
| 785 | # Text::xSV::Slurp cannot override column names, so skip slurp when | |||||
| 786 | # column_names is set â the DBI CSV connection will supply names instead. | |||||
| 787 | 84 | 62232 | if(((-s $slurp_file) <= $max_slurp_size) && !$params->{'column_names'}) { | |||
| 788 | 72 | 295 | if((-s $slurp_file) == 0) { | |||
| 789 | # Empty file | |||||
| 790 | 0 | 0 | $self->{'data'} = (); | |||
| 791 | } else { | |||||
| 792 | 72 | 3833 | require Text::xSV::Slurp; | |||
| 793 | 72 | 58705 | Text::xSV::Slurp->import(); | |||
| 794 | ||||||
| 795 | 72 | 148 | $self->_debug('slurp in'); | |||
| 796 | ||||||
| 797 | 72 | 2291 | my $dataref = xsv_slurp( | |||
| 798 | shape => 'aoh', | |||||
| 799 | text_csv => { | |||||
| 800 | sep_char => $sep_char, | |||||
| 801 | allow_loose_quotes => 1, | |||||
| 802 | blank_is_undef => 1, | |||||
| 803 | empty_is_undef => 1, | |||||
| 804 | binary => 1, | |||||
| 805 | escape_char => '\\', | |||||
| 806 | }, | |||||
| 807 | # string => \join('', grep(!/^\s*(#|$)/, <DATA>)) | |||||
| 808 | file => $slurp_file | |||||
| 809 | ); | |||||
| 810 | ||||||
| 811 | # Filter out blank lines and comment rows (lines starting with #) | |||||
| 812 | 72 311 335 72 | 13391 397 346 90 | my @data = grep { $_->{$self->{'id'}} !~ /^\s*#/ } grep { defined($_->{$self->{'id'}}) } @{$dataref}; | |||
| 813 | ||||||
| 814 | 72 | 130 | if($self->{'no_entry'}) { | |||
| 815 | # Not keyed on a primary column â keep as ordered list. | |||||
| 816 | # Only store a reference when rows were found; an empty-array ref | |||||
| 817 | # is truthy, which would activate the in-memory fast-path and | |||||
| 818 | # silently return 0 results instead of falling through to SQL. | |||||
| 819 | 8 | 24 | $self->{'data'} = @data ? \@data : undef; | |||
| 820 | } else { | |||||
| 821 | # Key the hash by $self->{'id'} for O(1) entry lookups | |||||
| 822 | 64 250 | 66 398 | $self->{'data'} = { map { $_->{$self->{'id'}} => $_ } @data }; | |||
| 823 | } | |||||
| 824 | } | |||||
| 825 | } | |||||
| 826 | 84 | 233 | $self->{'type'} = 'CSV'; | |||
| 827 | } else { | |||||
| 828 | 12 | 41 | $slurp_file = File::Spec->catfile($dir, "$dbname.xml"); | |||
| 829 | 12 | 62 | if(-r $slurp_file) { | |||
| 830 | 10 | 36 | if((-s $slurp_file) <= $max_slurp_size) { | |||
| 831 | 7 | 363 | require XML::Simple; | |||
| 832 | 7 | 3828 | XML::Simple->import(); | |||
| 833 | ||||||
| 834 | 7 | 248 | my $xml = XMLin($slurp_file); | |||
| 835 | 7 7 | 71620 11 | my @keys = keys %{$xml}; | |||
| 836 | 7 | 9 | my $key = $keys[0]; | |||
| 837 | 7 | 5 | my @data; | |||
| 838 | 7 | 26 | if(ref($xml->{$key}) eq 'ARRAY') { | |||
| 839 | 3 3 | 4 5 | @data = @{$xml->{$key}}; | |||
| 840 | } elsif(ref($xml) eq 'ARRAY') { | |||||
| 841 | 0 0 | 0 0 | @data = @{$xml}; | |||
| 842 | } elsif((ref($xml) eq 'HASH') && !$self->{'no_entry'}) { | |||||
| 843 | 3 3 | 3 4 | if(scalar(keys %{$xml}) == 1) { | |||
| 844 | 2 | 3 | if($xml->{$table}) { | |||
| 845 | 1 | 2 | @data = $xml->{$table}; | |||
| 846 | } else { | |||||
| 847 | 1 | 13 | Carp::croak('XML slurp: complex documents with an "entry" field are not yet supported'); | |||
| 848 | } | |||||
| 849 | } else { | |||||
| 850 | 1 | 6 | Carp::croak('XML slurp: multi-key documents are not yet supported'); | |||
| 851 | } | |||||
| 852 | } else { | |||||
| 853 | 1 | 11 | Carp::croak('XML slurp: cannot handle ', ref($xml), ' structure'); | |||
| 854 | } | |||||
| 855 | 4 | 8 | $self->{'data'} = (); | |||
| 856 | 4 | 12 | if($self->{'no_entry'}) { | |||
| 857 | # Not keyed, will need to scan each entry | |||||
| 858 | 1 | 1 | my $i = 0; | |||
| 859 | 1 | 2 | foreach my $d(@data) { | |||
| 860 | 5 | 7 | $self->{'data'}->{$i++} = $d; | |||
| 861 | } | |||||
| 862 | } else { | |||||
| 863 | # keyed on the $self->{'id'} (default: "entry") column | |||||
| 864 | 3 | 4 | foreach my $d(@data) { | |||
| 865 | 11 | 19 | $self->{'data'}->{$d->{$self->{'id'}}} = $d; | |||
| 866 | } | |||||
| 867 | } | |||||
| 868 | } else { | |||||
| 869 | 3 | 11 | $dbh = DBI->connect('dbi:XMLSimple(RaiseError=>1):'); | |||
| 870 | 3 | 187673 | $dbh->{'RaiseError'} = 1; | |||
| 871 | 3 | 38 | $self->_debug("read in $table from XML $slurp_file"); | |||
| 872 | 3 | 127 | $dbh->func($table, 'XML', $slurp_file, 'xmlsimple_import'); | |||
| 873 | } | |||||
| 874 | } else { | |||||
| 875 | # throw Error(-file => "$dir/$table"); | |||||
| 876 | 2 | 9 | $self->_fatal("Can't find a file called '$dbname' for the table $table in $dir"); | |||
| 877 | } | |||||
| 878 | 7 | 39 | $self->{'type'} = 'XML'; | |||
| 879 | } | |||||
| 880 | } | |||||
| 881 | ||||||
| 882 | # ref() must be called on the variable, not on the result of 'eq' | |||||
| 883 | 103 68 | 403 197 | Data::Reuse::fixate(%{$self->{'data'}}) if($self->{'data'} && (ref($self->{'data'}) eq 'HASH')); | |||
| 884 | ||||||
| 885 | 103 | 10494 | $self->{$table} = $dbh; | |||
| 886 | 103 | 625 | my @statb = stat($slurp_file); | |||
| 887 | 103 | 192 | $self->{'_updated'} = $statb[9]; | |||
| 888 | ||||||
| 889 | 103 | 197 | return $self; | |||
| 890 | } | |||||
| 891 | ||||||
| 892 - 928 | =head2 selectall_arrayref
Returns a reference to an array of hash references for every row that
matches the given criteria, or C<undef> when there are no matches.
my $rows = $db->selectall_arrayref(); # all rows
my $rows = $db->selectall_arrayref(status => 'active'); # exact match
my $rows = $db->selectall_arrayref(score => { '>' => 8 }); # operator
The full criteria syntax is described in L</QUERY CRITERIA>.
Pass a C<join> key to combine with another table:
my $rows = $db->selectall_arrayref(
dept_name => 'Engineering',
join => { table => 'dept', on => 'e.dept_id = dept.id' },
);
Results are returned in the cache (if configured) and the returned array
reference is made read-only unless C<no_fixate> was set.
B<Note:> this always returns all matching rows. Use L</selectall_array>
in scalar context, or C<< $db->query->limit(1)->all() >>, to fetch just one row.
=head3 PSEUDOCODE
1. Parse criteria; extract and build any JOIN clause.
2. If data is slurped AND no joins AND criteria are simple:
a. No criteria -> return all rows as arrayref.
b. entry-only lookup -> return [$data{entry}].
c. Otherwise -> scan rows in-memory with _match_criterion.
3. Otherwise build SQL: SELECT * FROM table [JOIN] [WHERE] ORDER BY id.
4. Check cache; return cached arrayref on HIT.
5. prepare_cached + execute; fetch all rows.
6. Store result in cache; fixate the array; return arrayref.
=cut | |||||
| 929 | ||||||
| 930 | sub selectall_arrayref { | |||||
| 931 | 117 | 36210 | my $self = shift; | |||
| 932 | ||||||
| 933 | # Fire _open() first so $self->{'berkeley'} is known before we parse @_. | |||||
| 934 | # BerkeleyDB param parsing must use get_params(undef, \@_) so that | |||||
| 935 | # key-value pairs like (join => {...}) are not mangled by the positional | |||||
| 936 | # 'entry' mapping that non-BerkeleyDB paths use. | |||||
| 937 | 117 | 208 | $self->_open_table({}); | |||
| 938 | ||||||
| 939 | 117 | 102 | my $params; | |||
| 940 | ||||||
| 941 | 117 | 150 | if($self->{'berkeley'}) { | |||
| 942 | 1 | 3 | $params = Params::Get::get_params(undef, \@_) // {}; | |||
| 943 | 1 | 16 | return set_return($self->_scan_berkeley($params), { type => 'arrayref' }); | |||
| 944 | } | |||||
| 945 | ||||||
| 946 | 116 | 172 | if($self->{'no_entry'}) { | |||
| 947 | 75 | 115 | $params = Params::Get::get_params(undef, \@_); | |||
| 948 | } elsif(scalar(@_)) { | |||||
| 949 | 25 | 35 | $params = Params::Get::get_params('entry', @_); | |||
| 950 | } | |||||
| 951 | ||||||
| 952 | 116 | 1302 | my $table = $self->_open_table($params); | |||
| 953 | ||||||
| 954 | 116 | 154 | $params //= {}; | |||
| 955 | ||||||
| 956 | 116 | 85 | my $join_clause = ''; | |||
| 957 | 116 | 141 | if(my $join_spec = delete $params->{'join'}) { | |||
| 958 | 19 | 36 | $join_clause = $self->_build_joins($join_spec); | |||
| 959 | } | |||||
| 960 | ||||||
| 961 | 109 | 273 | if(!$join_clause && $self->{'data'} && !$self->_has_complex_criteria($params)) { | |||
| 962 | 27 27 | 19 51 | if(scalar(keys %{$params}) == 0) { | |||
| 963 | 11 | 35 | $self->_trace("$table: selectall_arrayref fast track return"); | |||
| 964 | 11 | 271 | if(ref($self->{'data'}) eq 'HASH') { | |||
| 965 | 10 10 | 16 26 | $self->_debug("$table: returning ", scalar keys %{$self->{'data'}}, ' entries'); | |||
| 966 | 10 10 | 206 22 | if(scalar keys %{$self->{'data'}} <= 10) { | |||
| 967 | 10 10 10 | 9 33 36 | $self->_debug(do { require Data::Dumper; Data::Dumper::Dumper($self->{'data'}) }); | |||
| 968 | } | |||||
| 969 | 10 10 | 204 21 | my @rc = values %{$self->{'data'}}; | |||
| 970 | 10 | 30 | return set_return(\@rc, { type => 'arrayref' }); | |||
| 971 | } | |||||
| 972 | 1 | 4 | return set_return($self->{'data'}, { type => 'arrayref'}); | |||
| 973 | 16 | 51 | } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) { | |||
| 974 | # exists() guard: fixate() locks all keys in the slurp hash; return [] | |||||
| 975 | # (not [undef]) when the key is missing so callers get an empty result | |||||
| 976 | return set_return([], { type => 'arrayref' }) | |||||
| 977 | 7 | 22 | unless exists($self->{'data'}->{$params->{'entry'}}); | |||
| 978 | 5 | 16 | return set_return([$self->{'data'}->{$params->{'entry'}}], { type => 'arrayref' }); | |||
| 979 | } elsif(ref($self->{'data'}) eq 'HASH') { | |||||
| 980 | # Scan in-memory hash for simple column criteria without touching DBI. | |||||
| 981 | # fixate() locks hash keys, so use exists() to avoid throwing on unknown columns. | |||||
| 982 | 9 | 21 | $self->_debug("$table: selectall_arrayref in-memory scan with criteria"); | |||
| 983 | my @rc = grep { | |||||
| 984 | 36 | 26 | my $row = $_; | |||
| 985 | 36 36 36 | 47 65 43 | all { $self->_match_criterion(exists($row->{$_}) ? $row->{$_} : undef, $params->{$_}) } keys %{$params} | |||
| 986 | 9 9 | 185 13 | } values %{$self->{'data'}}; | |||
| 987 | 9 | 19 | return set_return(\@rc, { type => 'arrayref' }); | |||
| 988 | } | |||||
| 989 | } | |||||
| 990 | ||||||
| 991 | 82 | 186 | my ($where, $wargs) = $self->_build_where($params); | |||
| 992 | 76 76 | 61 69 | my @query_args = @{$wargs}; | |||
| 993 | ||||||
| 994 | 76 | 59 | my $query = "SELECT * FROM $table"; | |||
| 995 | 76 | 83 | $query .= " $join_clause" if $join_clause; | |||
| 996 | 76 | 166 | if($join_clause) { | |||
| 997 | 11 | 12 | $query .= " WHERE $where" if $where; | |||
| 998 | } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||||
| 999 | 6 | 7 | my $id = $self->{'id'}; | |||
| 1000 | 6 | 10 | $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1001 | 6 | 10 | $query .= " AND ($where)" if $where; | |||
| 1002 | } else { | |||||
| 1003 | 59 | 63 | $query .= " WHERE $where" if $where; | |||
| 1004 | } | |||||
| 1005 | 76 | 101 | if(!$self->{'no_entry'}) { | |||
| 1006 | 10 | 13 | $query .= ' ORDER BY ' . $self->{'id'}; | |||
| 1007 | } | |||||
| 1008 | ||||||
| 1009 | 76 | 72 | if(defined($query_args[0])) { | |||
| 1010 | 50 | 108 | $self->_debug("selectall_arrayref $query: ", join(', ', @query_args)); | |||
| 1011 | } else { | |||||
| 1012 | 26 | 47 | $self->_debug("selectall_arrayref $query"); | |||
| 1013 | } | |||||
| 1014 | ||||||
| 1015 | 76 | 1611 | my $key; | |||
| 1016 | my $c; | |||||
| 1017 | 76 | 88 | if($c = $self->{cache}) { | |||
| 1018 | 0 | 0 | $key = ref($self) . "::$query array"; | |||
| 1019 | 0 | 0 | if(defined($query_args[0])) { | |||
| 1020 | 0 | 0 | $key .= ' ' . join(', ', @query_args); | |||
| 1021 | } | |||||
| 1022 | 0 | 0 | $self->_debug("cache key = '$key'"); | |||
| 1023 | 0 | 0 | if(my $rc = $c->get($key)) { | |||
| 1024 | 0 | 0 | $self->_debug('cache HIT'); | |||
| 1025 | 0 | 0 | return $rc; # We stored a ref to the array | |||
| 1026 | ||||||
| 1027 | # This use of a temporary variable is to avoid | |||||
| 1028 | # "Implicit scalar context for array in return" | |||||
| 1029 | # my @rc = @{$rc}; | |||||
| 1030 | # return @rc; | |||||
| 1031 | } | |||||
| 1032 | 0 | 0 | $self->_debug('cache MISS'); | |||
| 1033 | } else { | |||||
| 1034 | 76 | 85 | $self->_debug('cache not used'); | |||
| 1035 | } | |||||
| 1036 | ||||||
| 1037 | 76 | 1344 | if(my $sth = $self->{$table}->prepare_cached($query)) { | |||
| 1038 | 76 | 16152 | $sth->execute(@query_args) || croak("$query: @query_args"); | |||
| 1039 | ||||||
| 1040 | 76 | 24359 | my $rc; | |||
| 1041 | 76 | 703 | while(my $href = $sth->fetchrow_hashref()) { | |||
| 1042 | 205 205 205 | 1369 968 217 | push @{$rc}, $href if(scalar keys %{$href}); | |||
| 1043 | } | |||||
| 1044 | 76 | 330 | $c->set($key, $rc, $self->{'cache_duration'}) if $c; | |||
| 1045 | ||||||
| 1046 | 76 | 98 | if(!$self->{'no_fixate'}) { | |||
| 1047 | # forget() clears stale addressâcanonical mappings from prior calls; | |||||
| 1048 | # fixate() then deduplicates values within this result set only. | |||||
| 1049 | # Without forget(), freed hashref addresses from previous fixate calls | |||||
| 1050 | # can collide with new DBI hashrefs and return wrong canonical rows. | |||||
| 1051 | 76 | 136 | Data::Reuse::forget(); | |||
| 1052 | 76 76 | 580 94 | Data::Reuse::fixate(@{$rc}); | |||
| 1053 | } | |||||
| 1054 | ||||||
| 1055 | 76 | 11881 | return $rc; | |||
| 1056 | } | |||||
| 1057 | 0 | 0 | $self->_warn("selectall_arrayref failure on $query: @query_args"); | |||
| 1058 | 0 | 0 | croak("$query: @query_args"); | |||
| 1059 | } | |||||
| 1060 | ||||||
| 1061 - 1066 | =head2 selectall_hashref Deprecated alias for L</selectall_arrayref>. Use C<selectall_arrayref> in new code. =cut | |||||
| 1067 | ||||||
| 1068 | sub selectall_hashref | |||||
| 1069 | { | |||||
| 1070 | 7 | 1261 | my $self = shift; | |||
| 1071 | 7 | 542 | return $self->selectall_arrayref(@_); | |||
| 1072 | } | |||||
| 1073 | ||||||
| 1074 - 1087 | =head2 selectall_array Similar to L</selectall_arrayref> but returns a list of hash references rather than a reference to an array. my @rows = $db->selectall_array(status => 'active'); In B<scalar context> it applies C<LIMIT 1> and returns just the first matching hash reference - making it more efficient than C<selectall_arrayref> when you only need one row. In B<list context> all matching rows are returned. Accepts the same criteria and C<join> parameter as L</selectall_arrayref>. =cut | |||||
| 1088 | ||||||
| 1089 | sub selectall_array | |||||
| 1090 | { | |||||
| 1091 | 21 | 2951 | my $self = shift; | |||
| 1092 | ||||||
| 1093 | 21 | 60 | $self->_open_table({}); | |||
| 1094 | ||||||
| 1095 | 21 | 59 | if($self->{'berkeley'}) { | |||
| 1096 | 1 | 2 | my $params = Params::Get::get_params(undef, \@_) // {}; | |||
| 1097 | 1 | 11 | my $rows = $self->_scan_berkeley($params); | |||
| 1098 | 1 1 | 2 2 | return wantarray ? @{$rows} : $rows->[0]; | |||
| 1099 | } | |||||
| 1100 | ||||||
| 1101 | 20 | 42 | my $params = Params::Get::get_params(undef, \@_); | |||
| 1102 | 20 | 260 | my $table = $self->_open_table($params); | |||
| 1103 | ||||||
| 1104 | 20 | 43 | $params //= {}; | |||
| 1105 | 20 | 48 | my $join_clause = ''; | |||
| 1106 | 20 | 39 | if(my $join_spec = delete $params->{'join'}) { | |||
| 1107 | 1 | 2 | $join_clause = $self->_build_joins($join_spec); | |||
| 1108 | } | |||||
| 1109 | ||||||
| 1110 | 20 | 68 | if(!$join_clause && $self->{'data'} && !$self->_has_complex_criteria($params)) { | |||
| 1111 | 14 14 | 12 25 | if(scalar(keys %{$params}) == 0) { | |||
| 1112 | 8 | 19 | $self->_trace("$table: selectall_array fast track return"); | |||
| 1113 | 8 | 161 | if(ref($self->{'data'}) eq 'HASH') { | |||
| 1114 | 7 7 | 8 19 | return values %{$self->{'data'}}; | |||
| 1115 | } | |||||
| 1116 | 1 1 | 1 3 | return @{$self->{'data'}}; | |||
| 1117 | 6 | 34 | } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) { | |||
| 1118 | # exists() guard: fixate() locks all keys; return empty list (not undef) | |||||
| 1119 | # for a missing entry so callers in list context get 0 elements not 1 | |||||
| 1120 | 2 | 7 | return () unless exists($self->{'data'}->{$params->{'entry'}}); | |||
| 1121 | 1 | 3 | return $self->{'data'}->{$params->{'entry'}}; | |||
| 1122 | } elsif(ref($self->{'data'}) eq 'HASH') { | |||||
| 1123 | # Same as selectall_arrayref scan but returns a list | |||||
| 1124 | 4 | 9 | $self->_debug("$table: selectall_array in-memory scan with criteria"); | |||
| 1125 | my @rc = grep { | |||||
| 1126 | 16 | 9 | my $row = $_; | |||
| 1127 | 16 16 16 | 25 30 18 | all { $self->_match_criterion(exists($row->{$_}) ? $row->{$_} : undef, $params->{$_}) } keys %{$params} | |||
| 1128 | 4 4 | 87 6 | } values %{$self->{'data'}}; | |||
| 1129 | 4 | 11 | return @rc; | |||
| 1130 | } | |||||
| 1131 | } | |||||
| 1132 | ||||||
| 1133 | 6 | 15 | my ($where, $wargs) = $self->_build_where($params); | |||
| 1134 | 6 6 | 6 9 | my @query_args = @{$wargs}; | |||
| 1135 | ||||||
| 1136 | 6 | 8 | my $query = "SELECT * FROM $table"; | |||
| 1137 | 6 | 10 | $query .= " $join_clause" if $join_clause; | |||
| 1138 | 6 | 17 | if($join_clause) { | |||
| 1139 | 1 | 2 | $query .= " WHERE $where" if $where; | |||
| 1140 | } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||||
| 1141 | 1 | 1 | my $id = $self->{'id'}; | |||
| 1142 | 1 | 2 | $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1143 | 1 | 1 | $query .= " AND ($where)" if $where; | |||
| 1144 | } else { | |||||
| 1145 | 4 | 11 | $query .= " WHERE $where" if $where; | |||
| 1146 | } | |||||
| 1147 | 6 | 10 | if(!$self->{'no_entry'}) { | |||
| 1148 | 4 | 6 | $query .= ' ORDER BY ' . $self->{'id'}; | |||
| 1149 | } | |||||
| 1150 | 6 | 10 | if(!wantarray) { | |||
| 1151 | 2 | 1 | $query .= ' LIMIT 1'; | |||
| 1152 | } | |||||
| 1153 | ||||||
| 1154 | 6 | 9 | if(defined($query_args[0])) { | |||
| 1155 | 2 | 6 | $self->_debug("selectall_array $query: ", join(', ', @query_args)); | |||
| 1156 | } else { | |||||
| 1157 | 4 | 7 | $self->_debug("selectall_array $query"); | |||
| 1158 | } | |||||
| 1159 | ||||||
| 1160 | 6 | 154 | my $key; | |||
| 1161 | my $c; | |||||
| 1162 | 6 | 25 | if($c = $self->{cache}) { | |||
| 1163 | 0 | 0 | $key = ref($self) . '::' . $query; | |||
| 1164 | 0 | 0 | if(wantarray) { | |||
| 1165 | 0 | 0 | $key .= ' array'; | |||
| 1166 | } | |||||
| 1167 | 0 | 0 | if(defined($query_args[0])) { | |||
| 1168 | 0 | 0 | $key .= ' ' . join(', ', @query_args); | |||
| 1169 | } | |||||
| 1170 | 0 | 0 | $self->_debug("cache key = '$key'"); | |||
| 1171 | 0 | 0 | if(my $rc = $c->get($key)) { | |||
| 1172 | 0 | 0 | $self->_debug('cache HIT'); | |||
| 1173 | 0 0 | 0 0 | return wantarray ? @{$rc} : $rc; # We stored a ref to the array | |||
| 1174 | ||||||
| 1175 | # This use of a temporary variable is to avoid | |||||
| 1176 | # "Implicit scalar context for array in return" | |||||
| 1177 | # my @rc = @{$rc}; | |||||
| 1178 | # return @rc; | |||||
| 1179 | } | |||||
| 1180 | 0 | 0 | $self->_debug('cache MISS'); | |||
| 1181 | } else { | |||||
| 1182 | 6 | 9 | $self->_debug('cache not used'); | |||
| 1183 | } | |||||
| 1184 | ||||||
| 1185 | 6 | 120 | if(my $sth = $self->{$table}->prepare_cached($query)) { | |||
| 1186 | 6 | 3071 | $sth->execute(@query_args) || croak("$query: @query_args"); | |||
| 1187 | ||||||
| 1188 | 6 | 6512 | my $rc; | |||
| 1189 | 6 | 60 | while(my $href = $sth->fetchrow_hashref()) { | |||
| 1190 | 26 | 511 | if(!wantarray) { | |||
| 1191 | # Scalar context: return just the first row; cache it too | |||||
| 1192 | 2 | 6 | $sth->finish(); | |||
| 1193 | 2 | 2 | $c->set($key, [$href], $self->{'cache_duration'}) if $c; | |||
| 1194 | 2 | 6 | return $href; | |||
| 1195 | } | |||||
| 1196 | 24 24 | 11 68 | push @{$rc}, $href; | |||
| 1197 | } | |||||
| 1198 | 4 | 63 | $c->set($key, $rc, $self->{'cache_duration'}) if $c; | |||
| 1199 | ||||||
| 1200 | 4 | 5 | if($rc) { | |||
| 1201 | 4 | 8 | if(!$self->{'no_fixate'}) { | |||
| 1202 | 4 | 8 | Data::Reuse::forget(); | |||
| 1203 | 4 4 | 22 7 | Data::Reuse::fixate(@{$rc}); | |||
| 1204 | } | |||||
| 1205 | 4 4 | 966 16 | return @{$rc}; | |||
| 1206 | } | |||||
| 1207 | 0 | 0 | return; | |||
| 1208 | } | |||||
| 1209 | 0 | 0 | $self->_warn("selectall_array failure on $query: @query_args"); | |||
| 1210 | 0 | 0 | croak("$query: @query_args"); | |||
| 1211 | } | |||||
| 1212 | ||||||
| 1213 - 1218 | =head2 selectall_hash Deprecated alias for L</selectall_array>. Use C<selectall_array> in new code. =cut | |||||
| 1219 | ||||||
| 1220 | sub selectall_hash | |||||
| 1221 | { | |||||
| 1222 | 8 | 2309 | my $self = shift; | |||
| 1223 | 8 | 50 | return $self->selectall_array(@_); | |||
| 1224 | } | |||||
| 1225 | ||||||
| 1226 - 1236 | =head2 count
Returns the number of rows matching the given criteria.
my $total = $db->count();
my $active = $db->count(status => 'active');
my $high = $db->count(score => { '>' => 90 });
Accepts the full criteria syntax described in L</QUERY CRITERIA>.
=cut | |||||
| 1237 | ||||||
| 1238 | sub count | |||||
| 1239 | { | |||||
| 1240 | 76 | 6708 | my $self = shift; | |||
| 1241 | ||||||
| 1242 | 76 | 164 | $self->_open_table({}); | |||
| 1243 | ||||||
| 1244 | 72 | 145 | if($self->{'berkeley'}) { | |||
| 1245 | 1 | 2 | my $params = Params::Get::get_params(undef, \@_) // {}; | |||
| 1246 | 1 1 | 10 3 | return scalar @{$self->_scan_berkeley($params)}; | |||
| 1247 | } | |||||
| 1248 | ||||||
| 1249 | 71 | 147 | my $params = Params::Get::get_params(undef, \@_); | |||
| 1250 | 71 | 879 | my $table = $self->_open_table($params); | |||
| 1251 | ||||||
| 1252 | 71 | 146 | if($self->{'data'}) { | |||
| 1253 | 43 43 | 34 70 | if(scalar(keys %{$params}) == 0) { | |||
| 1254 | 31 | 69 | $self->_trace("$table: count fast track return"); | |||
| 1255 | 31 | 744 | if(ref($self->{'data'}) eq 'HASH') { | |||
| 1256 | 28 28 | 21 74 | return scalar keys %{$self->{'data'}}; | |||
| 1257 | } | |||||
| 1258 | 3 3 | 3 10 | return scalar @{$self->{'data'}}; | |||
| 1259 | 12 | 64 | } elsif((scalar(keys %{$params}) == 1) && defined($params->{'entry'}) && !$self->{'no_entry'}) { | |||
| 1260 | # exists() guard: fixate() locks all keys in the slurp hash | |||||
| 1261 | 7 | 37 | return (exists($self->{'data'}->{$params->{'entry'}}) && $self->{'data'}->{$params->{'entry'}}) ? 1 : 0; | |||
| 1262 | } | |||||
| 1263 | } | |||||
| 1264 | ||||||
| 1265 | 33 | 92 | my ($where, $wargs) = $self->_build_where($params); | |||
| 1266 | 29 29 | 26 36 | my @query_args = @{$wargs}; | |||
| 1267 | ||||||
| 1268 | 29 | 25 | my $query; | |||
| 1269 | 29 | 97 | if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||
| 1270 | 12 | 16 | my $id = $self->{'id'}; | |||
| 1271 | 12 | 19 | $query = "SELECT COUNT(*) FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1272 | 12 | 24 | $query .= " AND ($where)" if $where; | |||
| 1273 | } elsif($self->{'no_entry'}) { | |||||
| 1274 | 13 | 17 | $query = "SELECT COUNT(*) FROM $table"; | |||
| 1275 | 13 | 24 | $query .= " WHERE $where" if $where; | |||
| 1276 | } else { | |||||
| 1277 | 4 | 9 | $query = "SELECT COUNT(" . $self->{'id'} . ") FROM $table"; | |||
| 1278 | 4 | 8 | $query .= " WHERE $where" if $where; | |||
| 1279 | } | |||||
| 1280 | ||||||
| 1281 | 29 | 49 | if(defined($query_args[0])) { | |||
| 1282 | 11 | 28 | $self->_debug("count $query: ", join(', ', @query_args)); | |||
| 1283 | } else { | |||||
| 1284 | 18 | 41 | $self->_debug("count $query"); | |||
| 1285 | } | |||||
| 1286 | ||||||
| 1287 | 29 | 624 | my $key; | |||
| 1288 | my $c; | |||||
| 1289 | 29 | 54 | if($c = $self->{'cache'}) { | |||
| 1290 | # Opportunistic: if a selectall_arrayref for the same criteria is already | |||||
| 1291 | # in cache, derive the count from that array rather than hitting the DB. | |||||
| 1292 | # The key is built to match what selectall_arrayref would store. | |||||
| 1293 | 0 | 0 | $key = ref($self) . '::' . $query; | |||
| 1294 | 0 | 0 | $key =~ s/COUNT\((.+?)\)/$1/; | |||
| 1295 | 0 | 0 | $key .= ' array'; | |||
| 1296 | 0 | 0 | if(defined($query_args[0])) { | |||
| 1297 | 0 | 0 | $key .= ' ' . join(', ', @query_args); | |||
| 1298 | } | |||||
| 1299 | 0 | 0 | if(my $rc = $c->get($key)) { | |||
| 1300 | 0 | 0 | $self->_debug('count: cache HIT (selectall array)'); | |||
| 1301 | 0 0 | 0 0 | return ref($rc) eq 'ARRAY' ? scalar @{$rc} : 0; | |||
| 1302 | } | |||||
| 1303 | 0 | 0 | $self->_debug('count: cache MISS'); | |||
| 1304 | } else { | |||||
| 1305 | 29 | 37 | $self->_debug('cache not used'); | |||
| 1306 | } | |||||
| 1307 | ||||||
| 1308 | 29 | 516 | if(my $sth = $self->{$table}->prepare_cached($query)) { | |||
| 1309 | 29 | 30438 | $sth->execute(@query_args) || croak("$query: @query_args"); | |||
| 1310 | ||||||
| 1311 | 29 | 19069 | my $count = $sth->fetchrow_arrayref()->[0]; | |||
| 1312 | 29 | 424 | $sth->finish(); | |||
| 1313 | ||||||
| 1314 | 29 | 129 | return $count; | |||
| 1315 | } | |||||
| 1316 | 0 | 0 | $self->_warn("count failure on $query: @query_args"); | |||
| 1317 | 0 | 0 | croak("$query: @query_args"); | |||
| 1318 | } | |||||
| 1319 | ||||||
| 1320 - 1344 | =head2 fetchrow_hashref
Returns a hash reference for the first row matching the given criteria,
or C<undef> when there is no match. Always applies C<LIMIT 1>.
my $row = $db->fetchrow_hashref(entry => 'key1');
my $row = $db->fetchrow_hashref(score => { '>=' => 10 });
When C<no_entry> is B<not> set you may pass a single bare value and it is
used as the C<entry> key:
my $row = $db->fetchrow_hashref('key1'); # same as entry => 'key1'
Accepts the full criteria syntax described in L</QUERY CRITERIA>, including
the C<join> parameter:
my $row = $db->fetchrow_hashref(
name => 'Alice',
join => { table => 'dept', on => 'e.dept_id = dept.id' },
);
Pass C<< table => $other_table >> to query a table other than the one
derived from the class name.
=cut | |||||
| 1345 | ||||||
| 1346 | sub fetchrow_hashref { | |||||
| 1347 | 44 | 11321 | my $self = shift; | |||
| 1348 | ||||||
| 1349 | 44 | 127 | $self->_trace('Entering fetchrow_hashref'); | |||
| 1350 | ||||||
| 1351 | 44 | 1014 | my $params; | |||
| 1352 | ||||||
| 1353 | 44 | 80 | if(!$self->{'no_entry'}) { | |||
| 1354 | 35 | 70 | $params = Params::Get::get_params('entry', @_); | |||
| 1355 | } else { | |||||
| 1356 | 9 | 19 | $params = Params::Get::get_params(undef, @_); | |||
| 1357 | } | |||||
| 1358 | ||||||
| 1359 | 44 | 669 | my $table = $self->_open_table($params); | |||
| 1360 | ||||||
| 1361 | # ::diag($self->{'type'}); | |||||
| 1362 | 44 31 | 131 146 | if($self->{'data'} && (!$self->{'no_entry'}) && (scalar keys(%{$params}) == 1) && defined($params->{'entry'}) && !$self->_has_complex_criteria($params)) { | |||
| 1363 | 25 | 38 | $self->_debug('Fast return from slurped data'); | |||
| 1364 | # Use exists() â fixate() locks the outer hash; accessing a missing key throws | |||||
| 1365 | 25 | 605 | return exists($self->{'data'}->{$params->{'entry'}}) ? $self->{'data'}->{$params->{'entry'}} : undef; | |||
| 1366 | } | |||||
| 1367 | ||||||
| 1368 | 19 | 38 | if($self->{'berkeley'}) { | |||
| 1369 | # print STDERR ">>>>>>>>>>>>\n"; | |||||
| 1370 | # ::diag(Data::Dumper->new([$self->{'berkeley'}])->Dump()); | |||||
| 1371 | 0 0 | 0 0 | if((!$self->{'no_entry'}) && (scalar keys(%{$params}) == 1) && defined($params->{'entry'})) { | |||
| 1372 | 0 | 0 | return { entry => $self->{'berkeley'}->{$params->{'entry'}} }; | |||
| 1373 | } | |||||
| 1374 | 0 | 0 | my $id = $self->{'id'}; | |||
| 1375 | 0 0 | 0 0 | if($self->{'no_entry'} && (scalar keys(%{$params}) == 1) && defined($id) && defined($params->{$id})) { | |||
| 1376 | 0 | 0 | if(my $rc = $self->{'berkeley'}->{$params->{$id}}) { | |||
| 1377 | 0 | 0 | return { $params->{$id} => $rc } # Return key->value as a hash pair | |||
| 1378 | } | |||||
| 1379 | 0 | 0 | return; | |||
| 1380 | } | |||||
| 1381 | 0 | 0 | Carp::croak(ref($self), ': fetchrow_hashref is meaningless on a NoSQL database'); | |||
| 1382 | } | |||||
| 1383 | ||||||
| 1384 | 19 | 54 | my $target = delete($params->{'table'}) // $table; | |||
| 1385 | 19 | 22 | my $join_spec = delete $params->{'join'}; | |||
| 1386 | 19 | 40 | my $join_clause = $join_spec ? $self->_build_joins($join_spec) : ''; | |||
| 1387 | 19 | 47 | my ($where, $wargs) = $self->_build_where($params); | |||
| 1388 | 17 17 | 15 22 | my @query_args = @{$wargs}; | |||
| 1389 | ||||||
| 1390 | 17 | 29 | my $query = "SELECT * FROM $target"; | |||
| 1391 | 17 | 22 | $query .= " $join_clause" if $join_clause; | |||
| 1392 | 17 | 67 | if($join_clause) { | |||
| 1393 | 1 | 1 | $query .= " WHERE $where" if $where; | |||
| 1394 | } elsif(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||||
| 1395 | 5 | 6 | my $id = $self->{'id'}; | |||
| 1396 | 5 | 5 | $query .= " WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1397 | 5 | 10 | $query .= " AND ($where)" if $where; | |||
| 1398 | } else { | |||||
| 1399 | 11 | 43 | $query .= " WHERE $where" if $where; | |||
| 1400 | } | |||||
| 1401 | 17 | 25 | $query .= ' LIMIT 1'; | |||
| 1402 | 17 | 55 | if(defined($query_args[0])) { | |||
| 1403 | 16 | 27 | my @call_details = caller(0); | |||
| 1404 | 16 | 166 | $self->_debug("fetchrow_hashref $query: ", join(', ', @query_args), | |||
| 1405 | ' called from ', $call_details[2], ' of ', $call_details[1]); | |||||
| 1406 | } else { | |||||
| 1407 | 1 | 2 | $self->_debug("fetchrow_hashref $query"); | |||
| 1408 | } | |||||
| 1409 | 17 | 324 | my $key = ref($self) . '::'; | |||
| 1410 | 17 | 65 | if(defined($query_args[0])) { | |||
| 1411 | 16 | 24 | if(wantarray) { | |||
| 1412 | 0 | 0 | $key .= 'array '; | |||
| 1413 | } | |||||
| 1414 | 16 | 33 | $key .= "fetchrow $query " . join(', ', @query_args); | |||
| 1415 | } else { | |||||
| 1416 | 1 | 0 | $key .= "fetchrow $query"; | |||
| 1417 | } | |||||
| 1418 | 17 | 16 | my $c; | |||
| 1419 | 17 | 32 | if($c = $self->{cache}) { | |||
| 1420 | 0 | 0 | if(my $rc = $c->get($key)) { | |||
| 1421 | 0 | 0 | if(wantarray) { | |||
| 1422 | 0 | 0 | if(ref($rc) eq 'ARRAY') { | |||
| 1423 | 0 0 | 0 0 | return @{$rc}; # We stored a ref to the array | |||
| 1424 | } | |||||
| 1425 | } else { | |||||
| 1426 | 0 | 0 | return $rc; | |||
| 1427 | } | |||||
| 1428 | } | |||||
| 1429 | } | |||||
| 1430 | ||||||
| 1431 | my $sth = $self->{$table}->prepare_cached($query) | |||||
| 1432 | 17 | 86 | or Carp::croak(ref($self), ": prepare failed: ", $self->{$table}->errstr()); | |||
| 1433 | 17 | 20731 | $sth->execute(@query_args) || croak("$query: @query_args"); | |||
| 1434 | 17 | 9335 | my $rc = $sth->fetchrow_hashref(); | |||
| 1435 | 17 | 376 | $sth->finish(); | |||
| 1436 | 17 | 49 | if($c) { | |||
| 1437 | 0 | 0 | if($rc) { | |||
| 1438 | 0 | 0 | $self->_debug("stash $key=>$rc in the cache for ", $self->{'cache_duration'}); | |||
| 1439 | 0 0 0 | 0 0 0 | $self->_debug("returns ", do { require Data::Dumper; Data::Dumper->new([$rc])->Dump() }); | |||
| 1440 | } else { | |||||
| 1441 | 0 | 0 | $self->_debug("Stash $key=>undef in the cache for ", $self->{'cache_duration'}); | |||
| 1442 | } | |||||
| 1443 | 0 | 0 | $c->set($key, $rc, $self->{'cache_duration'}); | |||
| 1444 | } | |||||
| 1445 | 17 | 51 | return $rc; | |||
| 1446 | } | |||||
| 1447 | ||||||
| 1448 - 1469 | =head2 execute Execute a raw SQL query on the underlying database. # Scalar context: returns the first row as a hashref my $row = $db->execute(query => 'SELECT * FROM foo WHERE id = 1'); # List context: returns all rows as a list of hashrefs my @rows = $db->execute(query => 'SELECT * FROM foo WHERE score > ?', args => [80]); The C<FROM E<lt>tableE<gt>> clause is appended automatically if omitted. On CSV tables without C<no_entry> it may help to add C<WHERE entry IS NOT NULL AND entry NOT LIKE '#%'> to filter comment rows. If the data have been slurped into memory this method still hits the actual database file directly. C<args> is an arrayref of bind values (see L<DBI/execute>). =cut | |||||
| 1470 | ||||||
| 1471 | sub execute | |||||
| 1472 | { | |||||
| 1473 | 18 | 4405 | my $self = shift; | |||
| 1474 | ||||||
| 1475 | 18 | 39 | if($self->{'berkeley'}) { | |||
| 1476 | 1 | 13 | Carp::croak(ref($self), ': execute is meaningless on a NoSQL database'); | |||
| 1477 | } | |||||
| 1478 | ||||||
| 1479 | 17 | 30 | my $args = Params::Get::get_params('query', @_); | |||
| 1480 | ||||||
| 1481 | # Ensure the 'query' parameter is provided | |||||
| 1482 | Carp::croak(__PACKAGE__, ': Usage: execute(query => $query)') | |||||
| 1483 | 15 | 190 | unless defined $args->{'query'}; | |||
| 1484 | ||||||
| 1485 | 15 | 28 | my $table = $self->_open_table($args); | |||
| 1486 | ||||||
| 1487 | 15 | 18 | my $query = $args->{'query'}; | |||
| 1488 | ||||||
| 1489 | # Append "FROM <table>" if missing | |||||
| 1490 | 15 | 38 | $query .= " FROM $table" unless $query =~ /\sFROM\s/i; | |||
| 1491 | ||||||
| 1492 | # Log the query if a logger is available | |||||
| 1493 | 15 | 34 | $self->_debug("execute $query"); | |||
| 1494 | ||||||
| 1495 | # Prepare and execute the query | |||||
| 1496 | 15 | 364 | my $sth = $self->{$table}->prepare_cached($query); | |||
| 1497 | # DBI->execute() takes a list; normalise args to an array whether it | |||||
| 1498 | # was passed as an arrayref ([30]) or a bare scalar/list (30). | |||||
| 1499 | 15 | 2342 | if(exists($args->{'args'})) { | |||
| 1500 | 9 5 | 18 6 | my @bind = ref($args->{'args'}) eq 'ARRAY' ? @{$args->{'args'}} : ($args->{'args'}); | |||
| 1501 | 9 | 157 | $sth->execute(@bind) or croak("$query: ", join(', ', @bind)); | |||
| 1502 | } else { | |||||
| 1503 | 6 | 110 | $sth->execute() or croak($query); | |||
| 1504 | } | |||||
| 1505 | ||||||
| 1506 | # Fetch the results | |||||
| 1507 | 15 | 2972 | my @results; | |||
| 1508 | 15 | 126 | while (my $row = $sth->fetchrow_hashref()) { | |||
| 1509 | 31 | 263 | unless(wantarray) { | |||
| 1510 | 3 | 11 | $sth->finish(); | |||
| 1511 | 3 | 7 | return $row; | |||
| 1512 | } | |||||
| 1513 | 28 | 118 | push @results, $row; | |||
| 1514 | } | |||||
| 1515 | ||||||
| 1516 | # Return all rows as an array in list context | |||||
| 1517 | 12 | 88 | return @results; | |||
| 1518 | } | |||||
| 1519 | ||||||
| 1520 - 1526 | =head2 updated Returns the Unix timestamp of the last database update (mtime for file-based backends, or the time of the most recent C<new()> call for DSN-based connections). =cut | |||||
| 1527 | ||||||
| 1528 | sub updated { | |||||
| 1529 | 4 | 564 | my $self = shift; | |||
| 1530 | ||||||
| 1531 | 4 | 11 | return $self->{'_updated'}; | |||
| 1532 | } | |||||
| 1533 | ||||||
| 1534 - 1555 | =head2 columns Returns an array reference of column names for the current table. my $cols = $db->columns(); # e.g. ['entry', 'name', 'score', 'status'] The column list is determined by the backend: =over 4 =item * B<Slurp mode> - sorted keys of the first row in memory. =item * B<SQLite / other DBI> - a zero-row C<SELECT *> exposes the driver's C<NAME> attribute. =item * B<BerkeleyDB> - always returns C<['entry', 'value']>. =back The result is cached inside the object after the first call. =cut | |||||
| 1556 | ||||||
| 1557 | sub columns { | |||||
| 1558 | 16 | 2595 | my $self = shift; | |||
| 1559 | ||||||
| 1560 | 16 | 55 | return $self->{'_columns'} if $self->{'_columns'}; | |||
| 1561 | ||||||
| 1562 | 12 | 36 | my $table = $self->_open_table({}); | |||
| 1563 | ||||||
| 1564 | 12 | 16 | my @cols; | |||
| 1565 | ||||||
| 1566 | 12 | 21 | if($self->{'berkeley'}) { | |||
| 1567 | 1 | 2 | return $self->{'_columns'} = ['entry', 'value']; | |||
| 1568 | } | |||||
| 1569 | ||||||
| 1570 | 11 | 23 | if(my $data = $self->{'data'}) { | |||
| 1571 | 7 | 19 | if(ref($data) eq 'HASH') { | |||
| 1572 | 7 7 | 7 11 | my ($first) = values %{$data}; | |||
| 1573 | 7 7 | 11 13 | @cols = sort keys %{$first} if $first; | |||
| 1574 | } | |||||
| 1575 | } else { | |||||
| 1576 | 4 | 24 | my $sth = $self->{$table}->prepare_cached("SELECT * FROM $table WHERE 1=0"); | |||
| 1577 | 4 | 3637 | $sth->execute(); | |||
| 1578 | 4 4 | 6214 22 | @cols = @{$sth->{NAME}}; | |||
| 1579 | 4 | 34 | $sth->finish(); | |||
| 1580 | } | |||||
| 1581 | ||||||
| 1582 | 11 | 45 | return $self->{'_columns'} = \@cols; | |||
| 1583 | } | |||||
| 1584 | ||||||
| 1585 - 1628 | =head2 schema
Returns a hash reference describing the schema of the current table.
Each key is a column name; each value is a hash reference with these keys:
=over 4
=item * C<type> - data type string (e.g. C<TEXT>, C<INTEGER>, C<REAL>)
=item * C<nullable> - C<1> if the column may be NULL, C<0> if NOT NULL
=item * C<default> - default value string, or C<undef>
=item * C<pk> - C<1> if this column is (part of) the primary key, C<0> otherwise
=back
my $schema = $db->schema();
for my $col (sort keys %{$schema}) {
my $info = $schema->{$col};
printf "%s %s %s\n",
$col,
$info->{type},
$info->{nullable} ? 'NULL' : 'NOT NULL';
}
The schema is determined by the backend:
=over 4
=item * B<SQLite> - C<PRAGMA table_info(table)>
=item * B<Other DBI drivers> - C<< $dbh->column_info(...) >>
=item * B<Slurp mode> - inferred from the first row (all columns typed as C<TEXT>)
=item * B<BerkeleyDB> - always returns C<entry> (pk) and C<value>
=back
The result is cached inside the object after the first call.
=cut | |||||
| 1629 | ||||||
| 1630 | sub schema { | |||||
| 1631 | 15 | 3817 | my $self = shift; | |||
| 1632 | ||||||
| 1633 | 15 | 61 | return $self->{'_schema'} if $self->{'_schema'}; | |||
| 1634 | ||||||
| 1635 | 11 | 23 | my $table = $self->_open_table({}); | |||
| 1636 | 11 | 16 | my %schema; | |||
| 1637 | ||||||
| 1638 | 11 | 21 | if($self->{'berkeley'}) { | |||
| 1639 | 1 | 4 | return $self->{'_schema'} = { | |||
| 1640 | entry => { type => 'TEXT', nullable => 0, default => undef, pk => 1 }, | |||||
| 1641 | value => { type => 'TEXT', nullable => 1, default => undef, pk => 0 }, | |||||
| 1642 | }; | |||||
| 1643 | } | |||||
| 1644 | ||||||
| 1645 | 10 | 21 | if(my $data = $self->{'data'}) { | |||
| 1646 | 6 | 14 | if(ref($data) eq 'HASH') { | |||
| 1647 | 6 6 | 7 10 | my ($first) = values %{$data}; | |||
| 1648 | 6 | 18 | if($first) { | |||
| 1649 | 6 | 8 | my $id = $self->{'id'}; | |||
| 1650 | 6 6 | 6 15 | for my $col (keys %{$first}) { | |||
| 1651 | 12 | 42 | $schema{$col} = { | |||
| 1652 | type => 'TEXT', | |||||
| 1653 | nullable => ($col eq $id ? 0 : 1), | |||||
| 1654 | default => undef, | |||||
| 1655 | pk => ($col eq $id ? 1 : 0), | |||||
| 1656 | }; | |||||
| 1657 | } | |||||
| 1658 | } | |||||
| 1659 | } | |||||
| 1660 | } else { | |||||
| 1661 | 4 | 40 | my $driver = $self->{$table}->{'Driver'}{'Name'} // ''; | |||
| 1662 | 4 | 23 | if($driver eq 'SQLite') { | |||
| 1663 | 3 | 11 | my $sth = $self->{$table}->prepare_cached("PRAGMA table_info($table)"); | |||
| 1664 | 3 | 163 | $sth->execute(); | |||
| 1665 | 3 | 47 | while(my $row = $sth->fetchrow_hashref()) { | |||
| 1666 | $schema{$row->{'name'}} = { | |||||
| 1667 | type => $row->{'type'}, | |||||
| 1668 | nullable => !$row->{'notnull'}, | |||||
| 1669 | default => $row->{'dflt_value'}, | |||||
| 1670 | 8 | 72 | pk => $row->{'pk'}, | |||
| 1671 | }; | |||||
| 1672 | } | |||||
| 1673 | 3 | 36 | $sth->finish(); | |||
| 1674 | } else { | |||||
| 1675 | 1 | 12 | my $sth = $self->{$table}->column_info(undef, undef, $table, '%'); | |||
| 1676 | 1 | 2 | if($sth) { | |||
| 1677 | 0 | 0 | while(my $row = $sth->fetchrow_hashref()) { | |||
| 1678 | $schema{$row->{'COLUMN_NAME'}} = { | |||||
| 1679 | type => $row->{'TYPE_NAME'}, | |||||
| 1680 | nullable => $row->{'NULLABLE'}, | |||||
| 1681 | 0 | 0 | default => $row->{'COLUMN_DEF'}, | |||
| 1682 | pk => 0, | |||||
| 1683 | }; | |||||
| 1684 | } | |||||
| 1685 | 0 | 0 | $sth->finish(); | |||
| 1686 | } | |||||
| 1687 | } | |||||
| 1688 | } | |||||
| 1689 | ||||||
| 1690 | 10 | 33 | return $self->{'_schema'} = \%schema; | |||
| 1691 | } | |||||
| 1692 | ||||||
| 1693 - 1714 | =head2 query
Returns a new L<Database::Abstraction::Query> builder object bound to this
database instance, for fluent method-chaining queries.
# All active rows with high scores, newest first, max 10
my $rows = $db->query
->where(status => 'active')
->where(score => { '>' => 80 })
->order_by('score DESC')
->limit(10)
->all();
# Single row
my $row = $db->query->where(name => 'Alice')->first();
# Just a count
my $n = $db->query->where(status => 'active')->count();
See L<Database::Abstraction::Query> for the full API.
=cut | |||||
| 1715 | ||||||
| 1716 | sub query | |||||
| 1717 | { | |||||
| 1718 | 57 | 17003 | my $self = shift; | |||
| 1719 | 57 | 789 | require Database::Abstraction::Query; | |||
| 1720 | 57 | 103 | return Database::Abstraction::Query->new(_db => $self); | |||
| 1721 | } | |||||
| 1722 | ||||||
| 1723 - 1767 | =head2 AUTOLOAD - column shortcut
Calling an unknown method whose name matches a column name performs a column
lookup. The method name is the column you want; the arguments are criteria.
# Scalar context: return the first match
my $name = $db->name(entry => 'key1');
# List context: return all matching values
my @names = $db->name();
# Shortcut when the table has an 'entry' key column
my $name = $db->name('key1'); # same as name(entry => 'key1')
# Unique/distinct values
my @statuses = $db->status(distinct => 1);
B<In list context> the full column is returned (all rows), ordered by the
column value. B<In scalar context> only the first match is returned
(C<LIMIT 1>).
Results come from the slurp cache when available.
Throws an error if the column does not exist (slurp mode) or if AUTOLOAD
has been disabled with C<< auto_load => 0 >>.
=head3 PSEUDOCODE
1. Extract column name from $AUTOLOAD; guard on DESTROY.
2. Croak if auto_load => 0.
3. Validate $column against /^[a-zA-Z_][a-zA-Z0-9_]*$/.
4. If data is slurped:
a. List context, no params -> map column over all rows (exists guard).
b. entry-only param -> direct hash lookup (exists guard).
c. No params, scalar -> first value in hash.
d. no_entry set -> scan array for matching key/value pair.
e. Other params -> scan keyed hash for matching column.
5. If not slurped, build SQL:
- List: SELECT column FROM table [WHERE ...] ORDER BY column
- Scalar: SELECT DISTINCT column FROM table [WHERE ...] LIMIT 1
6. Check cache; return on HIT.
7. prepare_cached + execute; fetch result.
8. Store in cache; fixate; return.
=cut | |||||
| 1768 | ||||||
| 1769 | sub AUTOLOAD { | |||||
| 1770 | 82 | 12972 | our $AUTOLOAD; | |||
| 1771 | 82 | 331 | my ($column) = $AUTOLOAD =~ /::(\w+)$/; | |||
| 1772 | ||||||
| 1773 | 82 | 141 | return if($column eq 'DESTROY'); | |||
| 1774 | ||||||
| 1775 | 82 | 144 | my $self = shift or return; | |||
| 1776 | ||||||
| 1777 | 82 | 110 | Carp::croak(__PACKAGE__, ": Unknown column $column") if(!ref($self)); | |||
| 1778 | ||||||
| 1779 | # Allow the AUTOLOAD feature to be disabled | |||||
| 1780 | 82 | 783 | Carp::croak(__PACKAGE__, ": AUTOLOAD disabled (auto_load => 0)") if(exists($self->{'auto_load'}) && !$self->{'auto_load'}); | |||
| 1781 | ||||||
| 1782 | # Validate column name - only allow safe column name | |||||
| 1783 | 78 | 195 | Carp::croak(__PACKAGE__, ": Invalid column name: $column") unless $column =~ /^[a-zA-Z_][a-zA-Z0-9_]*$/; | |||
| 1784 | ||||||
| 1785 | 78 | 1353 | my $table = $self->_open_table(); | |||
| 1786 | ||||||
| 1787 | 77 | 61 | my %params; | |||
| 1788 | 77 | 171 | if(ref($_[0]) eq 'HASH') { | |||
| 1789 | 2 2 | 2 5 | %params = %{$_[0]}; | |||
| 1790 | } elsif((scalar(@_) % 2) == 0) { | |||||
| 1791 | 53 | 81 | %params = @_; | |||
| 1792 | } elsif(scalar(@_) == 1) { | |||||
| 1793 | # Don't error on key-value databases, since there's no idea of columns | |||||
| 1794 | 22 | 58 | if($self->{'no_entry'} && !$self->{'berkeley'}) { | |||
| 1795 | 0 | 0 | Carp::croak(ref($self), "::($_[0]): ", $self->{'id'}, ' is not a column'); | |||
| 1796 | } | |||||
| 1797 | 22 | 31 | $params{'entry'} = shift; | |||
| 1798 | } | |||||
| 1799 | ||||||
| 1800 | 77 | 122 | if($self->{'berkeley'}) { | |||
| 1801 | 0 | 0 | if(my $id = $self->{'id'}) { | |||
| 1802 | 0 | 0 | return $self->{'berkeley'}->{$params{$id}}; | |||
| 1803 | } | |||||
| 1804 | 0 | 0 | return $self->{'berkeley'}->{$params{'entry'}}; | |||
| 1805 | } | |||||
| 1806 | ||||||
| 1807 | 77 | 109 | croak('Where did the data come from?') if(!defined($self->{'type'})); | |||
| 1808 | 77 | 60 | my $query; | |||
| 1809 | 77 | 82 | my $done_where = 0; | |||
| 1810 | 77 | 171 | my $distinct = delete($params{'distinct'}) || delete($params{'unique'}); | |||
| 1811 | ||||||
| 1812 | 77 | 129 | if(wantarray && !$distinct) { | |||
| 1813 | 18 | 53 | if(((scalar keys %params) == 0) && (my $data = $self->{'data'})) { | |||
| 1814 | # Return all column values from the in-memory hash. | |||||
| 1815 | # Use exists() because fixate() locks inner row hashes â | |||||
| 1816 | # accessing a disallowed key would throw without the guard. | |||||
| 1817 | # Handle both HASH (keyed data) and ARRAY (no_entry CSV slurp). | |||||
| 1818 | 15 2 13 | 54 2 21 | my @_rows = ref($data) eq 'ARRAY' ? @{$data} : values %{$data}; | |||
| 1819 | 15 62 | 20 94 | return map { exists($_->{$column}) ? $_->{$column} : undef } @_rows; | |||
| 1820 | } | |||||
| 1821 | 3 | 4 | my $id = $self->{'id'}; | |||
| 1822 | 3 | 9 | if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||
| 1823 | 2 | 3 | $query = "SELECT $column FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1824 | 2 | 3 | $done_where = 1; | |||
| 1825 | } else { | |||||
| 1826 | 1 | 2 | $query = "SELECT $column FROM $table"; | |||
| 1827 | } | |||||
| 1828 | } else { | |||||
| 1829 | 59 | 98 | if(my $data = $self->{'data'}) { | |||
| 1830 | # The data has been read in using Text::xSV::Slurp, | |||||
| 1831 | # so no need to do any SQL | |||||
| 1832 | 42 | 59 | $self->_debug('AUTOLOAD using slurped data'); | |||
| 1833 | 42 | 1074 | if($self->{'no_entry'}) { | |||
| 1834 | 2 | 3 | $self->_debug('no_entry is set'); | |||
| 1835 | 2 | 24 | my ($key, $value) = %params; | |||
| 1836 | 2 | 4 | if(defined($key)) { | |||
| 1837 | 2 | 5 | $self->_debug("key = $key, value = $value, column = $column"); | |||
| 1838 | 2 2 | 22 7 | foreach my $row(@{$data}) { | |||
| 1839 | # exists() guards: fixate() locks row hashes recursively | |||||
| 1840 | 2 | 16 | next unless exists($row->{$key}) && defined($row->{$key}) && $row->{$key} eq $value; | |||
| 1841 | 2 | 3 | my $rc = exists($row->{$column}) ? $row->{$column} : undef; | |||
| 1842 | 2 | 9 | $self->_trace(__LINE__, ": AUTOLOAD $key: return ", defined($rc) ? "'$rc'" : 'undef', ' from slurped data'); | |||
| 1843 | 2 | 26 | return $rc; | |||
| 1844 | } | |||||
| 1845 | 0 | 0 | $self->_debug('not found in slurped data'); | |||
| 1846 | } | |||||
| 1847 | } elsif(((scalar keys %params) == 1) && defined(my $key = $params{'entry'})) { | |||||
| 1848 | # Look up a single entry by its key. | |||||
| 1849 | # Use exists() before accessing â fixate() locks the outer hash and | |||||
| 1850 | # dereferencing a missing key on a locked hash throws an exception. | |||||
| 1851 | 30 | 24 | my $rc; | |||
| 1852 | 30 | 66 | if(exists($data->{$key}) && defined(my $hash = $data->{$key})) { | |||
| 1853 | 21 | 37 | if(!exists($hash->{$column})) { | |||
| 1854 | 3 | 27 | Carp::croak(__PACKAGE__, ": There is no column $column in $table"); | |||
| 1855 | } | |||||
| 1856 | 18 | 20 | $rc = $hash->{$column}; | |||
| 1857 | } | |||||
| 1858 | 27 | 36 | if(defined($rc)) { | |||
| 1859 | 17 | 45 | $self->_trace(__LINE__, ": AUTOLOAD $key: return '$rc' from slurped data"); | |||
| 1860 | } else { | |||||
| 1861 | 10 | 26 | $self->_trace(__LINE__, ": AUTOLOAD $key: return undef from slurped data"); | |||
| 1862 | } | |||||
| 1863 | 27 | 631 | return $rc | |||
| 1864 | } elsif((scalar keys %params) == 0) { | |||||
| 1865 | 7 | 9 | if(wantarray) { | |||
| 1866 | 7 | 10 | if($distinct) { | |||
| 1867 | 7 22 27 27 7 | 7 29 21 36 8 | my %h = map { $_ => 1 } grep { defined } map { exists($_->{$column}) ? $_->{$column} : undef } values %{$data}; | |||
| 1868 | 7 | 19 | return keys %h; | |||
| 1869 | } | |||||
| 1870 | # DEAD CODE: unreachable because the outer `if(wantarray && !$distinct)` | |||||
| 1871 | # handles the wantarray+!distinct case. In this else branch, wantarray | |||||
| 1872 | # implies $distinct (which returns above), so this line is never executed. | |||||
| 1873 | # return map { exists($_->{$column}) ? $_->{$column} : undef } values %{$data} | |||||
| 1874 | } | |||||
| 1875 | # Scalar: return the first value found without building a full list | |||||
| 1876 | 0 0 | 0 0 | foreach my $v (values %{$data}) { | |||
| 1877 | 0 | 0 | return exists($v->{$column}) ? $v->{$column} : undef; | |||
| 1878 | } | |||||
| 1879 | } else { | |||||
| 1880 | # Keyed data but filtering on a non-key column | |||||
| 1881 | 3 | 6 | my ($key, $value) = %params; | |||
| 1882 | 3 3 | 4 5 | foreach my $row (values %{$data}) { | |||
| 1883 | 6 | 23 | next unless exists($row->{$key}) && defined($row->{$key}) && $row->{$key} eq $value; | |||
| 1884 | 3 | 4 | next unless exists($row->{$column}); | |||
| 1885 | 3 | 10 | if(my $rc = $row->{$column}) { | |||
| 1886 | 3 | 8 | $self->_trace(__LINE__, ": AUTOLOAD $key: return '$rc' from slurped data"); | |||
| 1887 | 3 | 73 | return $rc | |||
| 1888 | } | |||||
| 1889 | } | |||||
| 1890 | } | |||||
| 1891 | return | |||||
| 1892 | 0 | 0 | } | |||
| 1893 | # Data has not been slurped in | |||||
| 1894 | 17 | 23 | my $id = $self->{'id'}; | |||
| 1895 | 17 | 47 | if(($self->{'type'} eq 'CSV') && !$self->{'no_entry'}) { | |||
| 1896 | 10 | 14 | $query = "SELECT DISTINCT $column FROM $table WHERE $id IS NOT NULL AND $id NOT LIKE '#%'"; | |||
| 1897 | 10 | 11 | $done_where = 1; | |||
| 1898 | } else { | |||||
| 1899 | 7 | 13 | $query = "SELECT DISTINCT $column FROM $table"; | |||
| 1900 | } | |||||
| 1901 | } | |||||
| 1902 | 20 | 33 | my @args; | |||
| 1903 | # Avoid `each` â it carries hidden iterator state across calls | |||||
| 1904 | 20 | 46 | for my $k (sort keys %params) { | |||
| 1905 | # Guard against SQL injection via column names â same rule as _build_where_conditions | |||||
| 1906 | 16 | 60 | Carp::croak(__PACKAGE__, ": unsafe column name '$k'") | |||
| 1907 | unless $k =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/; | |||||
| 1908 | 14 | 19 | my $value = $params{$k}; | |||
| 1909 | 14 | 40 | $self->_debug(__PACKAGE__, ": AUTOLOAD adding key/value pair $k=>", defined($value) ? $value : 'NULL'); | |||
| 1910 | 14 | 619 | if(defined($value)) { | |||
| 1911 | 12 | 30 | $query .= $done_where ? " AND $k = ?" : " WHERE $k = ?"; | |||
| 1912 | 12 | 14 | $done_where = 1; | |||
| 1913 | 12 | 21 | push @args, $value; | |||
| 1914 | } else { | |||||
| 1915 | 2 | 6 | $query .= $done_where ? " AND $k IS NULL" : " WHERE $k IS NULL"; | |||
| 1916 | 2 | 3 | $done_where = 1; | |||
| 1917 | } | |||||
| 1918 | } | |||||
| 1919 | 18 | 24 | if(wantarray) { | |||
| 1920 | 4 | 7 | $query .= " ORDER BY $column"; | |||
| 1921 | } else { | |||||
| 1922 | 14 | 17 | $query .= ' LIMIT 1'; | |||
| 1923 | } | |||||
| 1924 | 18 | 47 | if(scalar(@args) && $args[0]) { | |||
| 1925 | 12 | 31 | $self->_debug("AUTOLOAD $query: ", join(', ', @args)); | |||
| 1926 | } else { | |||||
| 1927 | 6 | 11 | $self->_debug("AUTOLOAD $query"); | |||
| 1928 | } | |||||
| 1929 | 18 | 698 | my $cache; | |||
| 1930 | 18 | 23 | my $key = ref($self) . '::'; | |||
| 1931 | 18 | 32 | if($cache = $self->{cache}) { | |||
| 1932 | 0 | 0 | if(wantarray) { | |||
| 1933 | 0 | 0 | $key .= 'array '; | |||
| 1934 | } | |||||
| 1935 | 0 | 0 | if(defined($args[0])) { | |||
| 1936 | 0 | 0 | $key .= "fetchrow $query " . join(', ', @args); | |||
| 1937 | } else { | |||||
| 1938 | 0 | 0 | $key .= "fetchrow $query"; | |||
| 1939 | } | |||||
| 1940 | 0 | 0 | if(my $rc = $cache->get($key)) { | |||
| 1941 | 0 | 0 | $self->_debug('cache HIT'); | |||
| 1942 | 0 0 | 0 0 | return wantarray ? @{$rc} : $rc; # We stored a ref to the array | |||
| 1943 | } | |||||
| 1944 | 0 | 0 | $self->_debug('cache MISS'); | |||
| 1945 | } else { | |||||
| 1946 | 18 | 27 | $self->_debug('cache not used'); | |||
| 1947 | } | |||||
| 1948 | 18 | 588 | my $sth = $self->{$table}->prepare_cached($query) || croak($query); | |||
| 1949 | 18 | 30120 | $sth->execute(@args) || croak($query); | |||
| 1950 | ||||||
| 1951 | 18 | 27323 | if(wantarray) { | |||
| 1952 | 4 14 4 | 5 248 23 | my @rc = map { $_->[0] } @{$sth->fetchall_arrayref()}; | |||
| 1953 | 4 | 8 | if($cache) { | |||
| 1954 | 0 | 0 | $cache->set($key, \@rc, $self->{'cache_duration'}); # Store a ref to the array | |||
| 1955 | } | |||||
| 1956 | 4 | 14 | Data::Reuse::fixate(@rc) if(!$self->{'no_fixate'}); | |||
| 1957 | 4 | 239 | return @rc; | |||
| 1958 | } | |||||
| 1959 | 14 | 89 | my $rc = $sth->fetchrow_array(); # Return the first match only | |||
| 1960 | 14 | 249 | $sth->finish(); | |||
| 1961 | 14 | 80 | if($cache) { | |||
| 1962 | # Store the value, then return it â cache->set() return value is unreliable | |||||
| 1963 | 0 | 0 | $cache->set($key, $rc, $self->{'cache_duration'}); | |||
| 1964 | } | |||||
| 1965 | 14 | 49 | return $rc; | |||
| 1966 | } | |||||
| 1967 | ||||||
| 1968 | sub DESTROY | |||||
| 1969 | { | |||||
| 1970 | 189 | 59757 | if(defined($^V) && ($^V ge 'v5.14.0')) { | |||
| 1971 | 189 | 300 | return if ${^GLOBAL_PHASE} eq 'DESTRUCT'; # >= 5.14.0 only | |||
| 1972 | } | |||||
| 1973 | 189 | 266 | my $self = shift; | |||
| 1974 | ||||||
| 1975 | # Clean up temporary file â deleting the File::Temp object triggers auto-unlink | |||||
| 1976 | 189 | 191 | delete $self->{'_temp_fh'}; | |||
| 1977 | ||||||
| 1978 | # Clean up database handles | |||||
| 1979 | 189 | 385 | my $table_name = $self->{'table'} || ref($self); | |||
| 1980 | 189 | 488 | $table_name =~ s/.*:://; | |||
| 1981 | ||||||
| 1982 | 189 | 313 | if(my $dbh = delete $self->{$table_name}) { | |||
| 1983 | 110 | 1724 | $dbh->disconnect() if $dbh->can('disconnect'); | |||
| 1984 | 110 | 1900 | $dbh->finish() if $dbh->can('finish'); | |||
| 1985 | } | |||||
| 1986 | ||||||
| 1987 | # Clean up Berkeley DB | |||||
| 1988 | 189 | 258 | if($self->{'berkeley'}) { | |||
| 1989 | 6 | 7 | eval { | |||
| 1990 | 6 6 | 2 11 | untie %{$self->{'berkeley'}}; | |||
| 1991 | }; | |||||
| 1992 | 6 | 8 | delete $self->{'berkeley'}; | |||
| 1993 | } | |||||
| 1994 | ||||||
| 1995 | # Clear all other attributes to break potential circular references | |||||
| 1996 | 189 | 334 | foreach my $key (keys %$self) { | |||
| 1997 | 1826 | 3614 | delete $self->{$key}; | |||
| 1998 | } | |||||
| 1999 | } | |||||
| 2000 | ||||||
| 2001 | # Build the JOIN clause(s) from a single join hashref or arrayref of hashrefs. | |||||
| 2002 | # Each spec needs keys: table (required), on (required), type (default INNER). | |||||
| 2003 | sub _build_joins | |||||
| 2004 | { | |||||
| 2005 | 31 | 1925 | my ($self, $join_spec) = @_; | |||
| 2006 | ||||||
| 2007 | 31 6 | 87 8 | my @specs = ref($join_spec) eq 'ARRAY' ? @{$join_spec} : ($join_spec); | |||
| 2008 | 31 155 | 36 147 | my %valid_types = map { $_ => 1 } qw(INNER LEFT RIGHT FULL CROSS); | |||
| 2009 | 31 | 28 | my @clauses; | |||
| 2010 | ||||||
| 2011 | 31 | 27 | for my $j (@specs) { | |||
| 2012 | 31 | 52 | my $type = uc($j->{'type'} // 'INNER'); | |||
| 2013 | 31 | 65 | my $jtable = $j->{'table'} or Carp::croak('join: missing "table"'); | |||
| 2014 | 28 | 61 | Carp::croak("join: unsafe table name '$jtable'") | |||
| 2015 | unless $jtable =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/; | |||||
| 2016 | 28 | 59 | my $on = $j->{'on'} or Carp::croak('join: missing "on" condition'); | |||
| 2017 | 25 | 69 | Carp::croak("Invalid JOIN type: $type") unless $valid_types{$type}; | |||
| 2018 | 21 | 30 | push @clauses, "$type JOIN $jtable ON ($on)"; | |||
| 2019 | } | |||||
| 2020 | ||||||
| 2021 | 21 | 44 | return join(' ', @clauses); | |||
| 2022 | } | |||||
| 2023 | ||||||
| 2024 | # Return true when $params contains operator hashrefs, -or, or -and groupings | |||||
| 2025 | # that the simple slurp fast-path cannot handle. | |||||
| 2026 | sub _has_complex_criteria | |||||
| 2027 | { | |||||
| 2028 | 74 | 90 | my ($self, $params) = @_; | |||
| 2029 | 74 | 110 | return 0 unless defined $params; | |||
| 2030 | 73 | 176 | return 1 if exists $params->{'-or'} || exists $params->{'-and'}; | |||
| 2031 | 71 71 | 50 117 | for my $v (values %{$params}) { | |||
| 2032 | 51 | 90 | return 1 if ref($v); | |||
| 2033 | } | |||||
| 2034 | 68 | 122 | return 0; | |||
| 2035 | } | |||||
| 2036 | ||||||
| 2037 | # Build the WHERE clause body (everything after "WHERE") from a criteria hash. | |||||
| 2038 | # Handles -or / -and groupings then delegates per-column work to _build_where_conditions. | |||||
| 2039 | # Returns ($sql_fragment, \@bind_values). | |||||
| 2040 | sub _build_where | |||||
| 2041 | { | |||||
| 2042 | 198 | 2089 | my ($self, $params) = @_; | |||
| 2043 | ||||||
| 2044 | 198 | 262 | $params //= {}; | |||
| 2045 | 198 198 | 165 230 | my %p = %{$params}; # work on a copy so we can delete -or/-and | |||
| 2046 | 198 | 172 | my @clauses; | |||
| 2047 | my @args; | |||||
| 2048 | ||||||
| 2049 | 198 | 249 | if(my $or_list = delete $p{'-or'}) { | |||
| 2050 | 11 | 9 | my (@sub_clauses, @sub_args); | |||
| 2051 | 11 11 | 8 12 | for my $cond (@{$or_list}) { | |||
| 2052 | 22 | 31 | my ($s, $a) = $self->_build_where_conditions($cond); | |||
| 2053 | 22 | 24 | if($s) { | |||
| 2054 | 22 | 24 | push @sub_clauses, "($s)"; | |||
| 2055 | 22 22 | 15 24 | push @sub_args, @{$a}; | |||
| 2056 | } | |||||
| 2057 | } | |||||
| 2058 | 11 | 15 | if(@sub_clauses) { | |||
| 2059 | 11 | 19 | push @clauses, '(' . join(' OR ', @sub_clauses) . ')'; | |||
| 2060 | 11 | 34 | push @args, @sub_args; | |||
| 2061 | } | |||||
| 2062 | } | |||||
| 2063 | 198 | 236 | if(my $and_list = delete $p{'-and'}) { | |||
| 2064 | 4 | 4 | my (@sub_clauses, @sub_args); | |||
| 2065 | 4 4 | 4 6 | for my $cond (@{$and_list}) { | |||
| 2066 | 7 | 8 | my ($s, $a) = $self->_build_where_conditions($cond); | |||
| 2067 | 7 | 7 | if($s) { | |||
| 2068 | 7 | 7 | push @sub_clauses, "($s)"; | |||
| 2069 | 7 7 | 6 9 | push @sub_args, @{$a}; | |||
| 2070 | } | |||||
| 2071 | } | |||||
| 2072 | 4 | 5 | if(@sub_clauses) { | |||
| 2073 | 4 | 5 | push @clauses, '(' . join(' AND ', @sub_clauses) . ')'; | |||
| 2074 | 4 | 6 | push @args, @sub_args; | |||
| 2075 | } | |||||
| 2076 | } | |||||
| 2077 | ||||||
| 2078 | 198 | 303 | my ($more, $margs) = $self->_build_where_conditions(\%p); | |||
| 2079 | 186 | 219 | if($more) { | |||
| 2080 | 100 | 73 | push @clauses, $more; | |||
| 2081 | 100 100 | 72 90 | push @args, @{$margs}; | |||
| 2082 | } | |||||
| 2083 | ||||||
| 2084 | 186 | 352 | return (join(' AND ', @clauses), \@args); | |||
| 2085 | } | |||||
| 2086 | ||||||
| 2087 | # Build a WHERE-body fragment for a flat col => val hash. | |||||
| 2088 | # Values may be plain scalars (= / LIKE / IS NULL) or operator hashrefs | |||||
| 2089 | # ({ '>' => n }, { -in => [...] }, { -between => [lo,hi] }, etc.). | |||||
| 2090 | sub _build_where_conditions | |||||
| 2091 | { | |||||
| 2092 | 241 | 3548 | my ($self, $params) = @_; | |||
| 2093 | ||||||
| 2094 | 241 | 157 | my @clauses; | |||
| 2095 | my @args; | |||||
| 2096 | ||||||
| 2097 | 241 241 | 179 344 | for my $col (sort keys %{$params}) { | |||
| 2098 | 161 | 144 | my $val = $params->{$col}; | |||
| 2099 | ||||||
| 2100 | # Guard against SQL injection via column names; allow table.column notation for JOINs | |||||
| 2101 | 161 | 389 | Carp::croak("_build_where_conditions: unsafe column name '$col'") | |||
| 2102 | unless $col =~ /^[a-zA-Z_][a-zA-Z0-9_.]*$/; | |||||
| 2103 | ||||||
| 2104 | 150 | 309 | if(ref($val) eq 'HASH') { | |||
| 2105 | 55 55 | 36 64 | for my $op (sort keys %{$val}) { | |||
| 2106 | 57 | 43 | my $operand = $val->{$op}; | |||
| 2107 | 57 | 195 | if($op eq '-in' || $op eq '-not_in') { | |||
| 2108 | 8 | 12 | my $sql_op = $op eq '-in' ? 'IN' : 'NOT IN'; | |||
| 2109 | 8 8 | 7 12 | my $ph = join(', ', ('?') x scalar(@{$operand})); | |||
| 2110 | 8 | 13 | push @clauses, "$col $sql_op ($ph)"; | |||
| 2111 | 8 8 | 6 10 | push @args, @{$operand}; | |||
| 2112 | } elsif($op eq '-between') { | |||||
| 2113 | 5 | 8 | push @clauses, "$col BETWEEN ? AND ?"; | |||
| 2114 | 5 | 10 | push @args, $operand->[0], $operand->[1]; | |||
| 2115 | } elsif($op eq '-like') { | |||||
| 2116 | 3 | 6 | push @clauses, "$col LIKE ?"; | |||
| 2117 | 3 | 6 | push @args, $operand; | |||
| 2118 | } elsif($op eq '-not_like') { | |||||
| 2119 | 3 | 5 | push @clauses, "$col NOT LIKE ?"; | |||
| 2120 | 3 | 6 | push @args, $operand; | |||
| 2121 | } elsif($op eq '!=') { | |||||
| 2122 | 4 | 5 | if(!defined($operand)) { | |||
| 2123 | 1 | 2 | push @clauses, "$col IS NOT NULL"; | |||
| 2124 | } else { | |||||
| 2125 | 3 | 5 | push @clauses, "$col != ?"; | |||
| 2126 | 3 | 4 | push @args, $operand; | |||
| 2127 | } | |||||
| 2128 | } elsif($op =~ /^(?:>|<|>=|<=)$/) { | |||||
| 2129 | 33 | 39 | push @clauses, "$col $op ?"; | |||
| 2130 | 33 | 45 | push @args, $operand; | |||
| 2131 | } else { | |||||
| 2132 | 1 | 8 | Carp::croak("Unknown operator '$op' for column '$col'"); | |||
| 2133 | } | |||||
| 2134 | } | |||||
| 2135 | } elsif(ref($val)) { | |||||
| 2136 | 3 | 20 | Carp::croak("$col: expected scalar or operator hashref, got ", ref($val)); | |||
| 2137 | } elsif(!defined($val)) { | |||||
| 2138 | 5 | 10 | push @clauses, "$col IS NULL"; | |||
| 2139 | } elsif($val =~ /[%_]/) { | |||||
| 2140 | 8 | 15 | push @clauses, "$col LIKE ?"; | |||
| 2141 | 8 | 11 | push @args, $val; | |||
| 2142 | } else { | |||||
| 2143 | 79 | 92 | push @clauses, "$col = ?"; | |||
| 2144 | 79 | 85 | push @args, $val; | |||
| 2145 | } | |||||
| 2146 | } | |||||
| 2147 | ||||||
| 2148 | 226 | 410 | return (join(' AND ', @clauses), \@args); | |||
| 2149 | } | |||||
| 2150 | ||||||
| 2151 | # Test a single in-memory row value against a criteria value. | |||||
| 2152 | # $crit_val may be a plain scalar or an operator hashref. | |||||
| 2153 | # Returns true when the row value satisfies the criterion. | |||||
| 2154 | # Scan the entire BerkeleyDB tied hash, building rows as {entry=>$k, value=>$v}, | |||||
| 2155 | # and filter by $params criteria using _match_criterion. | |||||
| 2156 | # Croaks when JOINs or -or/-and groupings are requested (unsupported for key-value stores). | |||||
| 2157 | sub _scan_berkeley | |||||
| 2158 | { | |||||
| 2159 | 3 | 3 | my ($self, $params) = @_; | |||
| 2160 | 3 | 5 | $params //= {}; | |||
| 2161 | ||||||
| 2162 | 3 | 6 | if(delete $params->{'join'}) { | |||
| 2163 | 0 | 0 | Carp::croak(ref($self), ': BerkeleyDB does not support JOINs'); | |||
| 2164 | } | |||||
| 2165 | 3 0 3 | 2 0 6 | if(grep { $_ eq '-or' || $_ eq '-and' } keys %{$params}) { | |||
| 2166 | 0 | 0 | Carp::croak(ref($self), ': BerkeleyDB does not support -or/-and groupings'); | |||
| 2167 | } | |||||
| 2168 | ||||||
| 2169 | 3 | 4 | my $bdb = $self->{'berkeley'}; | |||
| 2170 | 3 0 3 | 3 0 5 | my @rows = map { { entry => $_, value => $bdb->{$_} } } keys %{$bdb}; | |||
| 2171 | ||||||
| 2172 | 3 3 | 4 4 | if(my @cols = keys %{$params}) { | |||
| 2173 | @rows = grep { | |||||
| 2174 | 0 0 | 0 0 | my $row = $_; | |||
| 2175 | 0 | 0 | my $match = 1; | |||
| 2176 | 0 | 0 | for my $col (@cols) { | |||
| 2177 | 0 | 0 | unless($self->_match_criterion($row->{$col}, $params->{$col})) { | |||
| 2178 | 0 | 0 | $match = 0; | |||
| 2179 | 0 | 0 | last; | |||
| 2180 | } | |||||
| 2181 | } | |||||
| 2182 | 0 | 0 | $match; | |||
| 2183 | } @rows; | |||||
| 2184 | } | |||||
| 2185 | ||||||
| 2186 | 3 | 7 | return \@rows; | |||
| 2187 | } | |||||
| 2188 | ||||||
| 2189 | # SQL LIKE match using dynamic programming (O(m*n), no catastrophic backtracking). | |||||
| 2190 | # % matches any sequence of chars; _ matches exactly one char. Case-insensitive. | |||||
| 2191 | sub _like_match | |||||
| 2192 | { | |||||
| 2193 | 8 | 9 | my ($str, $pattern) = @_; | |||
| 2194 | 8 | 13 | my @s = split //, lc($str); | |||
| 2195 | 8 | 11 | my @p = split //, lc($pattern); | |||
| 2196 | 8 | 5 | my $m = scalar @s; | |||
| 2197 | 8 | 8 | my $n = scalar @p; | |||
| 2198 | ||||||
| 2199 | 8 38 | 16 39 | my @dp = map { [ (0) x ($m + 1) ] } 0 .. $n; | |||
| 2200 | 8 | 10 | $dp[0][0] = 1; | |||
| 2201 | ||||||
| 2202 | 8 | 12 | for my $i (1 .. $n) { | |||
| 2203 | 30 | 26 | if($p[$i - 1] eq '%') { | |||
| 2204 | 6 | 5 | $dp[$i][0] = $dp[$i - 1][0]; | |||
| 2205 | 6 | 6 | for my $j (1 .. $m) { | |||
| 2206 | 26 | 39 | $dp[$i][$j] = ($dp[$i - 1][$j] || $dp[$i][$j - 1]) ? 1 : 0; | |||
| 2207 | } | |||||
| 2208 | } else { | |||||
| 2209 | 24 | 22 | for my $j (1 .. $m) { | |||
| 2210 | 190 | 191 | $dp[$i][$j] = ($dp[$i - 1][$j - 1] | |||
| 2211 | && ($p[$i - 1] eq '_' || $p[$i - 1] eq $s[$j - 1])) ? 1 : 0; | |||||
| 2212 | } | |||||
| 2213 | } | |||||
| 2214 | } | |||||
| 2215 | 8 | 25 | return $dp[$n][$m]; | |||
| 2216 | } | |||||
| 2217 | ||||||
| 2218 | sub _match_criterion | |||||
| 2219 | { | |||||
| 2220 | 107 | 512 | my ($self, $row_val, $crit_val) = @_; | |||
| 2221 | ||||||
| 2222 | 107 | 109 | if(ref($crit_val) eq 'HASH') { | |||
| 2223 | 50 50 | 31 49 | for my $op (keys %{$crit_val}) { | |||
| 2224 | 50 | 43 | my $operand = $crit_val->{$op}; | |||
| 2225 | 50 | 110 | if($op eq '-in') { | |||
| 2226 | 5 10 4 | 7 15 5 | return 0 unless defined($row_val) && grep { $row_val eq $_ } @{$operand}; | |||
| 2227 | } elsif($op eq '-not_in') { | |||||
| 2228 | 4 10 4 | 7 14 5 | return 0 if defined($row_val) && grep { $row_val eq $_ } @{$operand}; | |||
| 2229 | } elsif($op eq '-between') { | |||||
| 2230 | 8 | 27 | return 0 unless defined($row_val) && $row_val >= $operand->[0] && $row_val <= $operand->[1]; | |||
| 2231 | } elsif($op eq '-like') { | |||||
| 2232 | 7 | 17 | return 0 unless defined($row_val); | |||
| 2233 | 6 | 12 | return 0 unless _like_match($row_val, $operand); | |||
| 2234 | } elsif($op eq '-not_like') { | |||||
| 2235 | 3 | 6 | return 0 unless defined($row_val); | |||
| 2236 | 2 | 4 | return 0 if _like_match($row_val, $operand); | |||
| 2237 | } elsif($op eq '!=') { | |||||
| 2238 | 8 | 9 | if(!defined($operand)) { | |||
| 2239 | 3 | 7 | return 0 unless defined($row_val); | |||
| 2240 | } else { | |||||
| 2241 | 5 | 13 | return 0 unless defined($row_val) && $row_val ne $operand; | |||
| 2242 | } | |||||
| 2243 | } elsif($op eq '>') { | |||||
| 2244 | 6 | 17 | return 0 unless defined($row_val) && $row_val > $operand; | |||
| 2245 | } elsif($op eq '<') { | |||||
| 2246 | 3 | 9 | return 0 unless defined($row_val) && $row_val < $operand; | |||
| 2247 | } elsif($op eq '>=') { | |||||
| 2248 | 3 | 8 | return 0 unless defined($row_val) && $row_val >= $operand; | |||
| 2249 | } elsif($op eq '<=') { | |||||
| 2250 | 3 | 8 | return 0 unless defined($row_val) && $row_val <= $operand; | |||
| 2251 | } | |||||
| 2252 | } | |||||
| 2253 | 26 | 51 | return 1; | |||
| 2254 | } | |||||
| 2255 | ||||||
| 2256 | 57 | 201 | return !defined($row_val) && !defined($crit_val) ? 1 | |||
| 2257 | : !defined($row_val) || !defined($crit_val) ? 0 | |||||
| 2258 | : $row_val eq $crit_val; | |||||
| 2259 | } | |||||
| 2260 | ||||||
| 2261 | # Determine the table and open the database | |||||
| 2262 | sub _open_table | |||||
| 2263 | { | |||||
| 2264 | 685 | 658 | my($self, $params) = @_; | |||
| 2265 | ||||||
| 2266 | # Get table name (remove package name prefix if present) | |||||
| 2267 | 685 | 2794 | my $table = $params->{'table'} || $self->{'table'} || ref($self); | |||
| 2268 | 685 | 1364 | $table =~ s/.*:://; | |||
| 2269 | ||||||
| 2270 | # Open a connection if it's not already open. | |||||
| 2271 | # BerkeleyDB never sets $self->{$table} (no DBI handle) or $self->{'data'}, | |||||
| 2272 | # so we also guard on $self->{'berkeley'} to avoid re-tying on every call. | |||||
| 2273 | 685 | 1318 | $self->_open() if((!$self->{$table}) && (!$self->{'data'}) && (!$self->{'berkeley'})); | |||
| 2274 | ||||||
| 2275 | 680 | 677 | return $table; | |||
| 2276 | } | |||||
| 2277 | ||||||
| 2278 | # Quote a SQL identifier using the current connection's dialect rules. | |||||
| 2279 | # Falls back to ANSI double-quoting when no connection is available. | |||||
| 2280 | sub _quote_identifier | |||||
| 2281 | { | |||||
| 2282 | 2 | 34 | my ($self, $name) = @_; | |||
| 2283 | ||||||
| 2284 | 2 | 6 | my $table = $self->{'table'} || ref($self); | |||
| 2285 | 2 | 5 | $table =~ s/.*:://; | |||
| 2286 | 2 | 4 | if(my $dbh = $self->{$table}) { | |||
| 2287 | 0 | 0 | return $dbh->quote_identifier($name); | |||
| 2288 | } | |||||
| 2289 | 2 | 6 | return qq{"$name"}; | |||
| 2290 | } | |||||
| 2291 | ||||||
| 2292 | # Determine whether a given file is a valid Berkeley DB file. | |||||
| 2293 | # It combines a fast preliminary check with a more thorough validation step for accuracy. | |||||
| 2294 | # It looks for the magic number at both byte 0 and byte 12 | |||||
| 2295 | # TODO: Combine _db_0 and _db_12 as they are very similar routines | |||||
| 2296 | sub _is_berkeley_db { | |||||
| 2297 | 98 | 602 | my ($self, $file) = @_; | |||
| 2298 | ||||||
| 2299 | # Step 1: Check magic number | |||||
| 2300 | # no autodie here: the file may not exist, and we want a silent false return | |||||
| 2301 | 98 | 104 | my $fh; | |||
| 2302 | 28 28 28 98 98 | 126 26 138 99 1391 | do { no autodie qw(open); open $fh, '<', $file } or return 0; | |||
| 2303 | 2 | 7 | binmode $fh; | |||
| 2304 | ||||||
| 2305 | 2 | 937 | my $is_db = (($self->_is_berkeley_db_0($fh)) || ($self->_is_berkeley_db_12($fh))); | |||
| 2306 | 2 | 26 | close $fh; | |||
| 2307 | ||||||
| 2308 | 2 | 362 | if($is_db) { | |||
| 2309 | # Step 2: Attempt to open as Berkeley DB | |||||
| 2310 | ||||||
| 2311 | 0 | 0 | require DB_File && DB_File->import(); | |||
| 2312 | ||||||
| 2313 | 0 | 0 | my %bdb; | |||
| 2314 | 0 | 0 | if(tie %bdb, 'DB_File', $file, O_RDONLY, 0644, $DB_File::DB_HASH) { | |||
| 2315 | # untie %db; | |||||
| 2316 | 0 | 0 | $self->{'berkeley'} = \%bdb; | |||
| 2317 | 0 | 0 | return 1; # Successfully identified as a Berkeley DB file | |||
| 2318 | } | |||||
| 2319 | } | |||||
| 2320 | 2 | 8 | return 0; | |||
| 2321 | } | |||||
| 2322 | ||||||
| 2323 | # Determine whether a given file is a valid Berkeley DB file. | |||||
| 2324 | # It combines a fast preliminary check with a more thorough validation step for accuracy. | |||||
| 2325 | sub _is_berkeley_db_0 | |||||
| 2326 | { | |||||
| 2327 | 3 | 229 | my ($self, $fh) = @_; | |||
| 2328 | ||||||
| 2329 | # Read the first 4 bytes (magic number) | |||||
| 2330 | 3 | 7 | read($fh, my $magic_bytes, 4) == 4 or return 0; | |||
| 2331 | ||||||
| 2332 | # Unpack both big-endian and little-endian values | |||||
| 2333 | 2 | 816 | my $magic_be = unpack('N', $magic_bytes); # Big-endian | |||
| 2334 | 2 | 4 | my $magic_le = unpack('V', $magic_bytes); # Little-endian | |||
| 2335 | ||||||
| 2336 | # Known Berkeley DB magic numbers (in both endian formats) | |||||
| 2337 | 2 8 | 4 11 | my %known_magic = map { $_ => 1 } ( | |||
| 2338 | 0x00061561, # Btree | |||||
| 2339 | 0x00053162, # Hash | |||||
| 2340 | 0x00042253, # Queue | |||||
| 2341 | 0x00052444, # Recno | |||||
| 2342 | ); | |||||
| 2343 | ||||||
| 2344 | 2 | 17 | return($known_magic{$magic_be} || $known_magic{$magic_le}); | |||
| 2345 | } | |||||
| 2346 | ||||||
| 2347 | sub _is_berkeley_db_12 | |||||
| 2348 | { | |||||
| 2349 | 3 | 297 | my ($self, $fh) = @_; | |||
| 2350 | 3 | 4 | my $header; | |||
| 2351 | ||||||
| 2352 | 3 | 5 | seek $fh, 12, 0 or return 0; | |||
| 2353 | 3 | 488 | read($fh, $header, 4) or return 0; | |||
| 2354 | ||||||
| 2355 | 2 | 58 | $header = substr(unpack('H*', $header), 0, 4); | |||
| 2356 | ||||||
| 2357 | # Berkeley DB magic numbers | |||||
| 2358 | 2 | 8 | return($header eq '6115' || $header eq '1561'); # Btree | |||
| 2359 | } | |||||
| 2360 | ||||||
| 2361 | # Log and remember a message | |||||
| 2362 | sub _log | |||||
| 2363 | { | |||||
| 2364 | 1078 | 1817 | my ($self, $level, @messages) = @_; | |||
| 2365 | ||||||
| 2366 | # FIXME: add caller's function | |||||
| 2367 | # if(($level eq 'warn') || ($level eq 'notice')) { | |||||
| 2368 | 1078 1078 | 778 2889 | push @{$self->{'messages'}}, { level => $level, message => join('', grep defined, @messages) }; | |||
| 2369 | # } | |||||
| 2370 | ||||||
| 2371 | 1078 | 2046 | if(scalar(@messages) && (my $logger = $self->{'logger'})) { | |||
| 2372 | 1078 | 2279 | $self->{'logger'}->$level(join('', grep defined, @messages)); | |||
| 2373 | } | |||||
| 2374 | } | |||||
| 2375 | ||||||
| 2376 | sub _debug { | |||||
| 2377 | 825 | 1272 | my $self = shift; | |||
| 2378 | 825 | 966 | $self->_log('debug', @_); | |||
| 2379 | } | |||||
| 2380 | ||||||
| 2381 | sub _trace { | |||||
| 2382 | 249 | 837 | my $self = shift; | |||
| 2383 | 249 | 408 | $self->_log('trace', @_); | |||
| 2384 | } | |||||
| 2385 | ||||||
| 2386 | # Emit a warning message somewhere | |||||
| 2387 | sub _warn { | |||||
| 2388 | 1 | 281 | my $self = shift; | |||
| 2389 | 1 | 2 | my $params = Params::Get::get_params('warning', \@_); | |||
| 2390 | ||||||
| 2391 | 1 | 16 | $self->_log('warn', $params->{'warning'}); | |||
| 2392 | 1 | 163 | Carp::carp(join('', grep defined, $params->{'warning'})); | |||
| 2393 | } | |||||
| 2394 | ||||||
| 2395 | # Die | |||||
| 2396 | sub _fatal { | |||||
| 2397 | 3 | 339 | my $self = shift; | |||
| 2398 | 3 | 6 | my $params = Params::Get::get_params('warning', \@_); | |||
| 2399 | ||||||
| 2400 | 3 | 51 | $self->_log('error', $params->{'warning'}); | |||
| 2401 | 3 | 739 | Carp::croak(join('', grep defined, $params->{'warning'})); | |||
| 2402 | } | |||||
| 2403 | ||||||
| 2404 - 2560 | =head1 AUTHOR Nigel Horne, C<< <njh at nigelhorne.com> >> =head1 SUPPORT This module is provided as-is without any warranty. Please report any bugs or feature requests to C<bug-database-abstraction at rt.cpan.org>, or through the web interface at L<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Database-Abstraction>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. =head1 MESSAGES The table below lists every error that the module can croak or carp, what triggers it, and how to resolve it. =over 4 =item C<< I<Class>: abstract class >> Direct instantiation of C<Database::Abstraction> was attempted. Create a subclass and instantiate that instead. =item C<< I<Class>: where are the files? >> Neither C<directory> nor C<dsn> was supplied to C<new()>. =item C<< I<Class>: I</path> is not a directory >> The C<directory> argument exists on disk but is not a directory. =item C<< I<Class>: cannot connect: I<$DBI::errstr> >> DBI failed to connect to the given C<dsn>. Check credentials and host. =item C<< Can't find a file called 'I<name>' for the table I<T> in I<dir> >> None of the probe extensions (C<.sql>, C<.psv>, C<.csv>, C<.db>, C<.xml>) matched in C<directory>. =item C<< I<Class>: prepare failed: I<$errstr> >> C<prepare_cached()> returned false. Usually a syntax error in an internally built query; file a bug if you see this from a normal API call. =item C<< _build_where_conditions: unsafe column name 'I<name>' >> A criteria key contained characters outside C<[A-Za-z0-9_.]>. This is a SQL-injection guard. Use only valid SQL identifier characters. =item C<< join: missing "table" >> / C<< join: missing "on" condition >> A join spec hashref is incomplete. Both C<table> and C<on> are required. =item C<< Invalid JOIN type: I<TYPE> >> C<type> in a join spec was not one of C<INNER LEFT RIGHT FULL CROSS>. =item C<< I<Class>: Unknown column I<col> >> / C<< I<Class>: AUTOLOAD disabled >> An AUTOLOAD call was made for a column that does not exist, or AUTOLOAD was disabled with C<< auto_load => 0 >>. =item C<< Usage: set_logger(logger => $logger) >> C<set_logger()> was called without a C<logger> argument. =item C<< Usage: execute(query => $query) >> C<execute()> was called without a C<query> argument. =item C<< XML slurp: I<...> is not yet supported >> The XML file structure is too complex for slurp mode. Use C<< max_slurp_size => 0 >> to force the DBI/XMLSimple SQL path. =item C<< I<Class>: I<method> is meaningless on a NoSQL database >> A relational method (C<selectall_arrayref>, C<count>, C<execute>, etc.) was called on a BerkeleyDB backend, which only supports key-value lookup via C<fetchrow_hashref>. =back =head1 KNOWN LIMITATIONS =over 4 =item * B<Read-only.> No INSERT, UPDATE, or DELETE is provided. C<execute()> runs raw read-only SQL. =item * B<Default CSV separator is C<!>>, not C<,>, for historical reasons. Pass C<< sep_char => ',' >> for standard RFC 4180 files. =item * B<Primary-key column is named C<entry>>, not C<key>, because C<key> is a SQL reserved word. Override with the C<id> parameter. =item * B<XML slurp is limited.> Only simple flat XML structures are supported in slurp mode. Multi-key or deeply nested documents will croak. Force SQL mode with C<< max_slurp_size => 0 >> if slurp fails. =item * B<Unique key assumption in slurp mode.> Duplicate values in the key column silently overwrite earlier rows. Disable slurp with C<< max_slurp_size => 0 >> if duplicates are expected. =item * B<BerkeleyDB does not support joins or the chained query builder.> =item * B<Column names must be valid SQL identifiers> (letters, digits, underscores, and a single dot for C<table.column> join notation). Other characters will cause a croak. =item * B<count() cache is opportunistic.> Count results are served from cache only when a prior C<selectall_arrayref()> or C<count()> call with the same criteria has already populated it. =back =head1 SEE ALSO =over 4 =item * L<Database::Abstraction::Query> - chained query builder =item * L<Configure an Object at Runtime|Object::Configure> =item * L<Test Dashboard|https://nigelhorne.github.io/Database-Abstraction/coverage/> =back =head1 LICENSE AND COPYRIGHT Copyright 2015-2026 Nigel Horne. Usage is subject to the GPL2 licence terms. If you use it, please let me know. =cut | |||||
| 2561 | ||||||
| 2562 | 1; | |||||