TER1 (Statement): 100.00%
TER2 (Branch): 93.75%
TER3 (LCSAJ): 100.0% (7/7)
Approximate LCSAJ segments: 49
● Covered — this LCSAJ path was executed during testing.
● Not covered — this LCSAJ path was never executed. These are the paths to focus on.
Multiple dots on a line indicate that multiple control-flow paths begin at that line. Hovering over any dot shows:
start → end → jump
Uncovered paths show [NOT COVERED] in the tooltip.
1: package Database::BI::Model::DataSource; 2: 3: use strict; 4: use warnings; 5: use autodie qw(:all); 6: 7: use Carp qw(croak carp); 8: use File::Spec (); 9: use Readonly; 10: use Scalar::Util qw(blessed); 11: use Sub::Protected; 12: use Params::Validate::Strict qw(validate_strict); 13: use Params::Get (); 14: 15: our $VERSION = '0.002.0'; 16: 17: # --------------------------------------------------------------------------- 18: # I18N message dictionary. 19: # All user-visible strings and exception messages are keyed here. 20: # To plug in a real i18n backend (e.g. Locale::Maketext), pass an object 21: # that responds to maketext($key, @args) as the "i18n" constructor argument; 22: # it will be called in preference to this table. 23: # --------------------------------------------------------------------------- 24: 25: Readonly our %MESSAGES => ( 26: error_directory_required => 'DataSource: argument "directory" is required', 27: error_table_required => 'DataSource: argument "table" is required', 28: error_directory_missing => 'DataSource: directory "%s" does not exist or is not readable', 29: error_table_name_invalid => 'DataSource: table name "%s" contains illegal characters (alphanumeric and underscore only)', 30: error_backend_init => 'DataSource: failed to initialise database backend for table "%s": %s', 31: error_fetch_failed => 'DataSource: fetch_all failed for table "%s": %s', 32: error_url_invalid => 'DataSource: URL "%s" must begin with http:// or https://', 33: error_url_fetch => 'DataSource: failed to open HTML table at "%s": %s', 34: warn_empty_result => 'DataSource: fetch_all returned no records for table "%s"', 35: warn_data_normalised => 'DataSource: result from backend was a hashref; converted to arrayref for table "%s"', 36: ); 37: 38: # A table name is a bare SQL-safe identifier: starts with a letter or 39: # underscore, followed by zero or more alphanumeric/underscore characters. 40: Readonly my $TABLE_NAME_RE => qr/\A[A-Za-z_][A-Za-z0-9_]*\z/; 41: 42: # --------------------------------------------------------------------------- 43: # Protected helpers 44: # --------------------------------------------------------------------------- 45: 46: # _url_label( $url ) -> $string 47: # 48: # Derive a safe, lowercase identifier from a URL for use as the table label. 49: # Takes the last non-empty path component, strips the extension, then replaces 50: # non-alphanumeric characters with underscores. Falls back to the hostname 51: # when the path component is absent or starts with a digit. 52: sub _url_label { ●53 → 61 → 73 53: my ($url) = @_; 54: my ($path) = $url =~ m{https?://[^/?#]+(.*)}i; 55: my @parts = grep { length } split m{/}, ($path // ''); 56: my $last = @parts ? $parts[-1] : ''; 57: # /s: . must cross \n in case a percent-decoded newline hides inside the URL. 58: $last =~ s/[?#].*//s; # strip query / fragment 59: $last =~ s/\.[^.]+\z//; # strip file extension (\z: no trailing-\n loophole) 60: $last =~ s/[^A-Za-z0-9_]/_/g; # sanitize 61: unless (length $last && $last =~ /\A[A-Za-z_]/) {Mutants (Total: 1, Killed: 1, Survived: 0)
62: # No usable path component -- fall back to hostname 63: my ($host) = $url =~ m{https?://([^/:?#]+)}; 64: $last = defined $host ? do { (my $h = $host) =~ s/[^A-Za-z0-9_]/_/g; $h } : 'html'; 65: } 66: # TODO: Unreachable code detected during path analysis. Investigate for removal. 67: # $last is always non-empty here: if the unless block did NOT fire, $last passed 68: # the "length $last && /\A[A-Za-z_]/" guard (non-empty by definition); if the 69: # unless block DID fire, $last was set to either the sanitized hostname (matched 70: # by [^/:?#]+, so >= 1 char, which stays >= 1 after s/[^A-Za-z0-9_]/_/g) or 71: # the literal 'html'. Either way, $last is always truthy, so 'html_table' can 72: # never be selected. The || fallback should be removed. 73: return lc($last || 'html_table');
Mutants (Total: 2, Killed: 2, Survived: 0)
74: } 75: 76: # _fmt( $key [, @sprintf_args] ) -> $string 77: # 78: # Package-level (not a method) i18n formatter. Looks up $key in %MESSAGES and 79: # applies sprintf if positional arguments are supplied. This function is used 80: # in new() before the object exists; instance methods should use _msg() instead 81: # so that a caller-supplied i18n object can override the built-in strings. 82: sub _fmt :Protected { 83: my ($key, @args) = @_; 84: my $tmpl = $MESSAGES{$key} // "Internal error: unknown message key '$key'"; 85: return @args ? sprintf($tmpl, @args) : $tmpl;
Mutants (Total: 2, Killed: 2, Survived: 0)
86: } 87: 88: # _msg( $self, $key [, @sprintf_args] ) -> $string 89: # 90: # Instance-level i18n formatter. Delegates to the caller-supplied i18n object 91: # (if any) before falling back to _fmt(). The i18n object must implement 92: # maketext($key, @args). 93: sub _msg :Protected { ●94 → 95 → 98 94: my ($self, $key, @args) = @_; 95: if (my $i18n = $self->{_i18n}) {
Mutants (Total: 1, Killed: 1, Survived: 0)
96: return $i18n->maketext($key, @args);
Mutants (Total: 2, Killed: 2, Survived: 0)
97: } 98: return _fmt($key, @args);
Mutants (Total: 2, Killed: 2, Survived: 0)
99: } 100: 101: # --------------------------------------------------------------------------- 102: # Constructor 103: # --------------------------------------------------------------------------- 104: 105: =head1 CONSTRUCTOR 106: 107: =head2 new 108: 109: Creates and returns a new C<Database::BI::Model::DataSource> instance. 110: 111: =head3 API SPECIFICATION 112: 113: =head4 INPUT 114: 115: { 116: directory => 'string', # required; path to the data directory 117: table => 'string', # required; bare table/file name (no extension) 118: i18n => { type => 'object', optional => 1, can => 'maketext' } # must implement maketext($key, @args) 119: } 120: 121: Accepts a flat key/value list, a hashref, or positional arguments via 122: C<Params::Get>. 123: 124: =head4 DOMAIN CONSTRAINTS 125: 126: =over 4 127: 128: =item C<directory> 129: 130: Must satisfy C<-d $directory> (must exist and be a directory). An empty 131: string, a non-existent path, or the path to a regular file all produce 132: C<error_directory_missing>. 133: 134: Valid partition: any existing directory path 135: Invalid partition: non-existent path, regular file path, empty string "" 136: 137: =item C<table> 138: 139: Must match C<TABLE_NAME_RE = \A[A-Za-z_][A-Za-z0-9_]*\z>. The first 140: character must be a letter (A-Z, a-z) or underscore; subsequent 141: characters may also be digits. 142: 143: Valid partition: "sales", "_tmp", "report_2024" (letter/underscore start) 144: Invalid partition: "1sales" (digit-start), "my.data" (dot), 145: "my-data" (hyphen), "" (empty string) 146: Boundary values: "a" (length-1 letter, valid), "_" (length-1 underscore, 147: valid), "1" (length-1 digit, croaks error_table_name_invalid) 148: 149: =back 150: 151: =head4 OUTPUT 152: 153: Returns C<$self> (a blessed hashref). Croaks on invalid arguments. 154: 155: =head3 MESSAGES 156: 157: error_directory_required -- "directory" argument was not supplied 158: error_table_required -- "table" argument was not supplied 159: error_directory_missing -- supplied directory does not exist / is unreadable 160: error_table_name_invalid -- table name fails the safe-identifier check 161: error_backend_init -- Database::Abstraction subclass could not be instantiated 162: 163: =head3 FORMAL SPECIFICATION 164: 165: new == [directory : PATH; table : NAME; i18n? : I18N_OBJECT] 166: pre (directory in dom FILE_SYSTEM /\ is_dir directory) 167: /\ table =~ TABLE_NAME_RE 168: post result.class = DataSource 169: /\ result._db.class = Database::Abstraction 170: 171: =cut 172: 173: sub new { 174: # Strategy: normalise the argument list with Params::Get so callers may 175: # pass a hashref or a flat list interchangeably, then validate strictly 176: # with Params::Validate before touching any value. 177: # When a "url" key is present, dispatch to the URL/HTML-table path instead. 178: my $class = shift; 179: my $raw = Params::Get::get_params(undef, \@_) // {}; 180: return $class->_new_from_url($raw) if exists $raw->{url};
Mutants (Total: 2, Killed: 2, Survived: 0)
181: 182: my $args = validate_strict( 183: schema => { 184: directory => { type => 'string' }, 185: table => { type => 'string' }, 186: i18n => { type => 'object', optional => 1, default => undef, can => 'maketext' }, 187: }, 188: input => $raw, 189: ); 190: 191: croak _fmt('error_directory_missing', $args->{directory}) 192: unless -d $args->{directory}; 193: 194: croak _fmt('error_table_name_invalid', $args->{table}) 195: unless $args->{table} =~ $TABLE_NAME_RE; 196: 197: my $self = bless { 198: _directory => $args->{directory}, 199: _table => lc $args->{table}, 200: _i18n => $args->{i18n}, 201: _db => undef, 202: }, $class; 203: 204: $self->_init_backend; 205: return $self;
Mutants (Total: 2, Killed: 2, Survived: 0)
206: 207: } 208: 209: # _new_from_url( $class, \%raw_params ) -> $self 210: # 211: # Purpose: Alternate constructor path for URL-backed HTML tables. 212: # Validates the URL scheme, derives a display label from the URL path, 213: # then calls _init_url_backend to build the D::A in-memory table. 214: # Entry: $raw->{url} must be an http:// or https:// URL. 215: # $raw->{html_table_index} (optional, default 0): zero-based table index. 216: # $raw->{i18n} (optional): Locale::Maketext-compatible object. 217: # Exit: Returns $self. Croaks on invalid URL scheme or backend init failure. 218: # Side Effects: Issues an HTTP GET to the URL via LWP::UserAgent. 219: sub _new_from_url :Protected { 220: my ($class, $raw) = @_; 221: 222: my $url = $raw->{url} // croak _fmt('error_url_invalid', ''); 223: croak _fmt('error_url_invalid', $url) 224: unless $url =~ m{\Ahttps?://}i; 225: 226: my $self = bless { 227: _url => $url, 228: _table => _url_label($url), 229: _i18n => $raw->{i18n}, 230: _id_col => undef, 231: _columns => undef, 232: _db => undef, 233: }, $class; 234: 235: $self->_init_url_backend($raw->{html_table_index} // 0); 236: return $self;
Mutants (Total: 2, Killed: 2, Survived: 0)
237: } 238: 239: # _init_url_backend( $self, $table_index ) -> void 240: # 241: # Purpose: Construct the Database::Abstraction backend for URL/HTML-table mode. 242: # D::A fetches the page via LWP::UserAgent, parses it with 243: # HTML::TableExtract, and stores all rows (including headers from row 0) 244: # as an in-memory arrayref. Column order is not recoverable after 245: # construction (hash keys lose order), so _columns stays undef and 246: # _get_columns in the controller falls back to alphabetical sorting. 247: # Entry: $self->{_url} is a valid http(s) URL; $table_index is a non-negative int. 248: # Exit: Sets $self->{_db}. Croaks on LWP or HTML::TableExtract failure. 249: # Side Effects: Network I/O; may take up to LWP's default timeout. 250: sub _init_url_backend :Protected { 251: my ($self, $table_index) = @_; 252: my $url = $self->{_url}; 253: 254: require Database::Abstraction; 255: 256: # Reuse a single generic package for all URL-backed instances. 257: # The package name is irrelevant for the URL code path -- D::A branches on 258: # the presence of $self->{'url'}, not on the class name. 259: my $pkg = 'Database::BI::_DB::HtmlUrl'; 260: { 261: no strict 'refs'; 262: push @{"${pkg}::ISA"}, 'Database::Abstraction' 263: unless $pkg->isa('Database::Abstraction'); 264: } 265: 266: my $db = eval { 267: $pkg->new({ 268: url => $url, 269: html_table_index => $table_index, 270: no_entry => 1, 271: }) 272: }; 273: croak $self->_msg('error_url_fetch', $url, $@) if $@; 274: 275: $self->{_db} = $db; 276: return; 277: } 278: 279: # --------------------------------------------------------------------------- 280: # Protected initialisation 281: # --------------------------------------------------------------------------- 282: 283: # _detect_file_info( $dir, $table ) -> \%info 284: # 285: # Peek at the first header line of a CSV or PSV file and return a hashref: 286: # sep_char => field separator character (',' or '|') 287: # id => first column name (used as Database::Abstraction's id key) 288: # columns => arrayref of all column names in file order 289: # 290: # Two non-obvious defaults in Database::Abstraction make this necessary: 291: # 1. sep_char defaults to '!' — so a plain CSV is read as one giant field 292: # per row, producing a single comma-joined string instead of columns. 293: # 2. id defaults to 'entry' — the slurp filter greps on that column; if it 294: # doesn't exist every row is silently discarded. 295: # Returns an empty hashref for non-CSV/PSV formats (SQLite, XML, etc.). 296: sub _detect_file_info :Protected { ●297 → 298 → 334 297: my ($dir, $table) = @_; 298: for my $ext (qw(csv psv)) { 299: my $path = File::Spec->catfile($dir, "$table.$ext"); 300: next unless -r $path; 301: # "use autodie" makes open() die on failure, so "or next" would be dead 302: # code. Disable autodie for this open so a vanishing file (race between 303: # the -r probe and the open) results in a clean skip rather than a croak. 304: my $fh; 305: { no autodie 'open'; open $fh, '<', $path or next } 306: my $line = <$fh>; 307: close $fh; 308: next unless defined $line; 309: chomp $line; 310: 311: my $sep; 312: if ($ext eq 'psv') {
Mutants (Total: 1, Killed: 1, Survived: 0)
313: $sep = '|'; 314: } else { 315: # Sniff the separator: Database::Abstraction uses '!' natively and 316: # sometimes stores those files with a .csv extension. If splitting 317: # on ',' yields a single field that itself contains '!', the real 318: # separator is almost certainly '!'. 319: my @probe = split /,/, $line, -1; 320: $sep = (@probe == 1 && $line =~ /!/) ? '!' : ',';
Mutants (Total: 1, Killed: 1, Survived: 0)
321: } 322: 323: my @cols = split /\Q$sep\E/, $line; 324: # Two separate substitutions: the /g on an anchored alternation wastes 325: # O(N) engine cycles retrying \A (which can only match at position 0). 326: for (@cols) { s/\A[\s"]+//; s/[\s"]+\z// } # strip whitespace and quotes 327: @cols = grep { length } @cols; 328: return { 329: sep_char => $sep, 330: id => (@cols ? $cols[0] : undef), 331: columns => \@cols, 332: }; 333: } 334: return {}; 335: } 336: 337: # _init_backend( $self ) -> void 338: # 339: # Strategy: Database::Abstraction is designed as a base class where the 340: # lowercased package name maps to the data file in the directory 341: # (e.g. "Database::BI::_DB::Sales" -> data/sales.csv or data/sales.db). 342: # We synthesise an ephemeral subclass at runtime so DataSource remains 343: # fully table-agnostic and callers never need to touch Database::Abstraction 344: # directly. In Phase 2, this method can be replaced with Database::Join 345: # instantiation without any change to the public API. 346: # 347: # no_entry => 1: a BI viewer wants every row; we do not need O(1) keyed 348: # lookups on a primary key. This stores data as an arrayref instead of a 349: # hashref, which the fast-track path in selectall_arrayref returns directly. 350: sub _init_backend :Protected { ●351 → 378 → 382 351: my $self = shift; 352: my $table = $self->{_table}; 353: my $dir = $self->{_directory}; 354: 355: require Database::Abstraction; 356: 357: my $pkg = 'Database::BI::_DB::' . ucfirst($table); 358: { 359: no strict 'refs'; 360: push @{"${pkg}::ISA"}, 'Database::Abstraction' 361: unless $pkg->isa('Database::Abstraction'); 362: } 363: 364: my $info = _detect_file_info($dir, $table); 365: my $id_col = $info->{id} // 'entry'; 366: $self->{_id_col} = $id_col; 367: $self->{_columns} = $info->{columns}; # undef for SQLite/XML 368: 369: my $db = eval { 370: $pkg->new({ 371: directory => $dir, 372: table => $table, 373: id => $id_col, 374: no_entry => 1, 375: defined($info->{sep_char}) ? (sep_char => $info->{sep_char}) : (), 376: }); 377: }; 378: if ($@) {
Mutants (Total: 1, Killed: 1, Survived: 0)
379: croak $self->_msg('error_backend_init', $table, $@); 380: } 381: 382: $self->{_db} = $db; 383: return; 384: } 385: 386: # --------------------------------------------------------------------------- 387: # Public accessors 388: # --------------------------------------------------------------------------- 389: 390: =head1 ACCESSORS 391: 392: =head2 table_name 393: 394: Returns the (lowercased) table name this instance was opened against. 395: 396: =head3 API SPECIFICATION 397: 398: =head4 INPUT 399: 400: None. 401: 402: =head4 OUTPUT 403: 404: Returns a C<SCALAR> string. 405: 406: =head3 MESSAGES 407: 408: None. 409: 410: =head3 FORMAL SPECIFICATION 411: 412: table_name == lambda self . self._table 413: 414: =cut 415: 416: sub table_name { 417: my $self = shift; 418: return $self->{_table};
Mutants (Total: 2, Killed: 2, Survived: 0)
419: } 420: 421: =head2 columns 422: 423: Returns an arrayref of column names in file order, or C<undef> when the 424: backend does not expose a fixed column order (e.g. SQLite, XML). 425: 426: =cut 427: 428: sub columns { 429: my $self = shift; 430: return $self->{_columns};
Mutants (Total: 2, Killed: 2, Survived: 0)
431: } 432: 433: =head2 id_column 434: 435: Returns the name of the column used as the primary key / slurp-filter anchor. 436: Returns C<undef> for URL/HTML-table backends (no primary-key concept applies). 437: 438: =cut 439: 440: sub id_column { 441: my $self = shift; 442: return $self->{_id_col};
Mutants (Total: 2, Killed: 2, Survived: 0)
443: } 444: 445: =head2 source_url 446: 447: Returns the source URL for URL/HTML-table-backed instances, or C<undef> for 448: file-backed instances. 449: 450: =cut 451: 452: sub source_url { 453: my $self = shift; 454: return $self->{_url};
Mutants (Total: 2, Killed: 2, Survived: 0)
455: } 456: 457: # --------------------------------------------------------------------------- 458: # Public data-access methods 459: # --------------------------------------------------------------------------- 460: 461: =head1 METHODS 462: 463: =head2 fetch_all 464: 465: Returns every record in the table as an arrayref of hashrefs. 466: 467: =head3 API SPECIFICATION 468: 469: =head4 INPUT 470: 471: None. Filtering is performed at the controller layer (C<Dashboard::_apply_filter_spec>) 472: after C<fetch_all> returns, not inside C<DataSource>. 473: 474: =head4 OUTPUT 475: 476: ARRAYREF of HASHREF # one hashref per row, keys are column names 477: # returns [] when the table exists but is empty 478: 479: Croaks if the backend raises an exception. Carps (non-fatal) when the result 480: set is empty so the caller can distinguish "open succeeded, no rows" from a 481: silent failure. 482: 483: =head3 MESSAGES 484: 485: error_fetch_failed -- backend threw an exception during retrieval 486: warn_empty_result -- query succeeded but returned zero records 487: warn_data_normalised -- backend returned a hashref; converted to arrayref 488: 489: =head3 FORMAL SPECIFICATION 490: 491: fetch_all == lambda self . 492: let rows = self._db.selectall_hashref() in 493: pre self._db /= undef 494: post result : seq HASHREF 495: /\ #result >= 0 496: 497: =cut 498: 499: sub fetch_all { ●500 → 504 → 508 500: my $self = shift; 501: my $table = $self->{_table}; 502: 503: my $data = eval { $self->{_db}->selectall_hashref() }; 504: if ($@) {
Mutants (Total: 1, Killed: 1, Survived: 0)
505: croak $self->_msg('error_fetch_failed', $table, $@); 506: } 507: ●508 → 514 → 519 508: return [] unless defined $data; 509: 510: # Strategy: Database::Abstraction can return either an arrayref (when the 511: # underlying driver iterates rows) or a hashref keyed by primary key (when 512: # it mirrors DBI's selectall_hashref semantics). We normalise to arrayref 513: # here so every caller above this layer sees a uniform structure. 514: if (ref $data eq 'HASH') {
Mutants (Total: 1, Killed: 1, Survived: 0)
515: carp $self->_msg('warn_data_normalised', $table); 516: $data = [ values %{$data} ]; 517: } 518: ●519 → 519 → 523 519: if (!@{$data}) {
Mutants (Total: 1, Killed: 1, Survived: 0)
520: carp $self->_msg('warn_empty_result', $table); 521: } 522: 523: return $data;
Mutants (Total: 2, Killed: 2, Survived: 0)
524: } 525: 526: 1; 527: 528: __END__ 529: 530: =head1 NAME 531: 532: Database::BI::Model::DataSource - Table-agnostic adapter around Database::Abstraction 533: 534: =head1 VERSION 535: 536: This document describes Database::BI::Model::DataSource version 0.01. 537: 538: =head1 SYNOPSIS 539: 540: B<Read all rows from a CSV file:> 541: 542: use Database::BI::Model::DataSource; 543: 544: my $source = Database::BI::Model::DataSource->new( 545: directory => '/path/to/data', 546: table => 'sales', # looks for data/sales.csv, .psv, .sql, .xml, etc. 547: ); 548: 549: my $records = $source->fetch_all; # arrayref of hashrefs -- one hashref per row 550: 551: for my $row (@{$records}) { 552: printf "Product: %s, Amount: %s\n", $row->{product}, $row->{amount}; 553: } 554: 555: B<Get column names in the original file order (CSV/PSV only):> 556: 557: my $cols = $source->columns; # returns undef for SQLite and XML 558: if ($cols) { 559: print join(', ', @{$cols}), "\n"; 560: } 561: 562: B<Find out which column is the primary key:> 563: 564: print "Primary key column: ", $source->id_column, "\n"; 565: 566: B<Open a SQLite file (.sql extension):> 567: 568: my $source = Database::BI::Model::DataSource->new( 569: directory => '/var/data', 570: table => 'inventory', # looks for /var/data/inventory.sql 571: ); 572: 573: B<Open a pipe-separated file (.psv extension):> 574: 575: my $source = Database::BI::Model::DataSource->new( 576: directory => '/var/data', 577: table => 'products', # looks for /var/data/products.psv 578: ); 579: 580: B<Use a custom i18n object to translate error messages:> 581: 582: # The i18n object must have a maketext($key, @args) method. 583: my $source = Database::BI::Model::DataSource->new( 584: directory => '/path/to/data', 585: table => 'sales', 586: i18n => My::I18N::Handle->new, 587: ); 588: 589: B<Handle errors gracefully:> 590: 591: my $source = eval { 592: Database::BI::Model::DataSource->new( 593: directory => $dir, 594: table => $table, 595: ); 596: }; 597: if ($@) { 598: carp "Could not open table: $@"; 599: # $@ contains a translated message from %MESSAGES 600: } 601: 602: my $records = eval { $source->fetch_all }; 603: if ($@) { 604: carp "Could not read records: $@"; 605: } 606: 607: =head1 DESCRIPTION 608: 609: C<Database::BI::Model::DataSource> is a thin, table-agnostic adapter that 610: wraps L<Database::Abstraction> and exposes three accessors (C<fetch_all>, 611: C<columns>, C<id_column>) used by the controller. 612: 613: L<Database::Abstraction> is a read-only ORM that discovers data files 614: (CSV, PSV, SQLite, XML, etc.) automatically from a directory based on the 615: calling class name. C<DataSource> generates an ephemeral subclass at 616: construction time so callers never interact with L<Database::Abstraction> 617: directly. To swap the backend for L<Database::Join> in Phase 2, only the 618: C<open_table> helper in C<Database::BI> needs to change; the controller and 619: C<DataSource> are untouched. 620: 621: C<_detect_file_info> peeks at the first header line of CSV/PSV files to 622: extract the correct separator character, the primary-key column name, and 623: the full ordered column list. Without this, two silent L<Database::Abstraction> 624: defaults corrupt every result: C<sep_char> defaults to C<'!'> (turning a 625: comma-separated file into a single-field table) and C<id> defaults to 626: C<'entry'> (causing every row to be discarded when no C<entry> column 627: exists). 628: 629: Result filtering (C<eq>, C<contains>, C<gt>, etc.) is performed at the 630: controller layer by C<Dashboard::_apply_filter_spec> after C<fetch_all> 631: returns. C<DataSource> itself is filter-unaware. 632: 633: All user-visible strings and exception messages are keyed through the 634: C<%MESSAGES> dictionary and routed via C<_msg()>, making every diagnostic 635: replaceable by an i18n object at instantiation time. 636: 637: =head1 COMMON PITFALLS 638: 639: These are the most common mistakes when using C<DataSource>. 640: 641: =over 4 642: 643: =item B<SQLite databases must use .sql as their file extension> 644: 645: C<Database::Abstraction> looks for SQLite databases with the C<.sql> extension. 646: It does B<not> recognise C<.sqlite>, C<.sqlite3>, or C<.db3>. If you have a 647: file called C<inventory.sqlite>, rename it to C<inventory.sql> before passing 648: it to C<DataSource>. 649: 650: =item B<The table name is always lowercased> 651: 652: The constructor lowercases the C<table> argument before using it. Passing 653: C<table =E<gt> 'Sales'> and C<table =E<gt> 'sales'> both look for 654: C<sales.csv> (or C<sales.sql>, etc.). The matching is case-insensitive on the 655: table name but B<case-sensitive on the directory path>. 656: 657: =item B<CSV files with the wrong separator appear as one giant field per row> 658: 659: C<Database::Abstraction> defaults to C<!> (exclamation mark) as its field 660: separator. A standard comma-separated CSV file will look like one big field 661: per row (for example, C<1,Widget A,North,100>) because the library never sees 662: the commas as separators. C<DataSource> fixes this automatically by reading 663: the first line of the file and detecting the actual separator. If you bypass 664: C<DataSource> and call C<Database::Abstraction> directly, you B<must> pass 665: C<sep_char =E<gt> ','> yourself. 666: 667: =item B<A table with no "entry" column returns zero rows (without DataSource)> 668: 669: C<Database::Abstraction> uses C<entry> as its default primary-key column name. 670: When slurping a CSV, it discards any row where C<$row-E<gt>{entry}> is 671: undefined. Because most CSV files do not have an C<entry> column, B<all rows 672: are silently discarded>. C<DataSource> prevents this by reading the actual 673: first column name from the CSV header and passing it as C<id>, and also by 674: setting C<no_entry =E<gt> 1> so all rows are kept as an ordered array. 675: 676: =item B<columns() returns undef for SQLite and XML files> 677: 678: C<columns()> only returns an arrayref for file formats where the header order 679: is visible before data is read (CSV and PSV). For SQLite and XML files it 680: returns C<undef>. Always check: C<if ($source-E<gt>columns) { ... }>. The 681: controller falls back to putting C<id_column> first and then sorting the rest 682: alphabetically when C<columns()> is C<undef>. 683: 684: =item B<XML elements named "name", "id", or "key" cause parse failures> 685: 686: C<XML::Simple> (used by C<Database::Abstraction> for XML files) automatically 687: turns a child element called C<name>, C<id>, or C<key> into a hash key instead 688: of keeping it as an array element. This breaks the expected data structure. 689: Use different element names in your XML: for example, C<E<lt>skuE<gt>>, 690: C<E<lt>labelE<gt>>, or C<E<lt>codeE<gt>> instead of C<E<lt>idE<gt>> and 691: C<E<lt>nameE<gt>>. 692: 693: <!-- WRONG: these element names trigger XMLin key-folding --> 694: <items> 695: <item><id>1</id><name>Widget</name></item> 696: </items> 697: 698: <!-- CORRECT: use neutral element names --> 699: <items> 700: <item><sku>1</sku><label>Widget</label></item> 701: </items> 702: 703: =item B<fetch_all returns an empty arrayref, not undef, for an empty table> 704: 705: When a table exists but contains no data rows, C<fetch_all> returns C<[]> (an 706: empty arrayref), not C<undef>. Check with C<scalar @{$records}>, not with 707: C<defined $records> or C<$records>. 708: 709: =back 710: 711: =head1 LIMITATIONS 712: 713: =over 4 714: 715: =item * 716: 717: Only read operations are supported. Write-back is not in scope. 718: 719: =item * 720: 721: One C<DataSource> instance corresponds to exactly one table. Multi-table 722: left joins are composed at the controller layer by C<Dashboard::_left_join>; 723: C<Database::Join> (Phase 2) is not yet in use. 724: 725: =item * 726: 727: The ephemeral backend class is generated into a package namespace 728: (C<Database::BI::_DB::*>) that persists for the lifetime of the process. 729: Instantiating two C<DataSource> objects for the same table name reuses 730: the same ephemeral class. 731: 732: =item * 733: 734: The C<i18n> object, if supplied, must implement C<maketext($key, @args)> 735: compatible with L<Locale::Maketext>. 736: 737: =back 738: 739: =head1 CONFIGURATION AND ENVIRONMENT 740: 741: No environment variables are read. All configuration is passed through 742: the constructor. 743: 744: =head1 DEPENDENCIES 745: 746: L<Carp>, L<Readonly>, L<Scalar::Util>, L<Params::Validate::Strict>, L<Params::Get>, 747: L<Database::Abstraction>. 748: 749: =head1 INCOMPATIBILITIES 750: 751: None known. 752: 753: =head1 BUGS AND LIMITATIONS 754: 755: Please report bugs via L<https://github.com/nigelhorne/Database-BI/issues>. 756: 757: =head1 AUTHOR 758: 759: Nigel Horne C<< <njh@nigelhorne.com> >> 760: 761: =head1 LICENCE AND COPYRIGHT 762: 763: Copyright 2026 Nigel Horne. 764: 765: Usage is subject to the GPL2 licence terms. 766: If you use it, 767: please let me know. 768: 769: =cut