File Coverage

=head1 NAME

Config::Abstraction - Merge and manage configuration data from different sources

=head1 VERSION

Version 0.40

=cut

=head1 SYNOPSIS

=head2 Pattern 1: Environment overrides file overrides in-code defaults

The most common pattern for twelve-factor-style apps.
The C<data> argument supplies defaults, a YAML file supplies site configuration,
and environment variables allow per-deployment overrides without touching any file.

  # config/base.yaml
  #   database:
  #     host: db.example.com
  #     port: 5432
  #     user: app

  use Config::Abstraction;

  my $config = Config::Abstraction->new(
      data        => { database => { host => 'localhost', port => 5432 } },
      config_dirs => ['config'],
      env_prefix  => 'APP_',
  );

  # In production, set APP_DATABASE__HOST=db.prod.example.com in the environment.
  # That silently overrides the file value, which in turn overrides the default.
  my $host = $config->get('database.host');
  my $port = $config->get('database.port');

=head2 Pattern 2: Command-line arguments override everything

Useful for CLI tools where operator flags must win over every other source.

  # Run as: myscript.pl --APP_LOGLEVEL=debug --APP_DATABASE__HOST=localhost

  use Config::Abstraction;

  my $config = Config::Abstraction->new(
      config_dirs => ['config'],
      env_prefix  => 'APP_',
  );

  # @ARGV is consumed and stripped during new(); the resulting config already
  # has cli-layer values at the highest precedence.
  my $loglevel = $config->get('loglevel');       # 'debug'  (from --APP_LOGLEVEL)
  my $db_host  = $config->get('database.host');  # 'localhost' (from --APP_DATABASE__HOST)

=head2 Pattern 3: Multi-file layering (base + local override)

Separates shared defaults from machine-specific tweaks.
Every developer has a C<base.yaml>; only the production server has C<local.yaml>.

  # config/base.yaml   -- checked into version control
  #   database:
  #     host: localhost
  #     user: dev
  #
  # config/local.yaml  -- NOT checked in; production only
  #   database:
  #     host: db.prod.example.com
  #     user: produser
  #     password: s3cr3t

  use Config::Abstraction;

  my $config = Config::Abstraction->new(
      config_dirs => ['config'],   # loads base.yaml then local.yaml
      env_prefix  => 'APP_',
  );

  # On a dev machine (no local.yaml): host='localhost', user='dev'
  # On the prod server:               host='db.prod.example.com', user='produser'
  my $db = $config->get('database.host');
  my $user = $config->get('database.user');

=head1 DESCRIPTION

C<Config::Abstraction> is a flexible configuration management layer that sits above C<Config::*> modules.
It provides a simple way to layer multiple configuration sources with predictable merge order.
It lets you define sources such as:

=over 4

=item * Perl hashes (in-memory defaults or dynamic values)

=item * Environment variables (with optional prefixes)

=item * Configuration files (YAML, JSON, INI, or plain key=value)

=item * Command-line arguments

=back

Sources are applied in the order they are provided. Later sources override
earlier ones unless a key is explicitly set to C<undef> in the later source.

In addition to using drivers to load configuration data from multiple file
formats (YAML, JSON, XML, and INI),
it also allows levels of configuration, each of which overrides the lower levels.
So, it also integrates environment variable
overrides and command line arguments for runtime configuration adjustments.
This module is designed to help developers manage layered configurations that can be loaded from files and overridden at run-time for debugging,
offering a modern, robust and dynamic approach
to configuration management.

=head2 Merge Precedence

Sources are applied in the order shown below.  Each row wins over every row
above it.  When the same key appears in multiple sources, the highest-priority
source always determines the final value - including when that value is C<undef>.

  Priority   Source                         Set with
  --------   ------                         --------
     1 (lo)  data constructor argument      data => { key => 'default' }
     2        base.*  config files          config/base.yaml, base.json, ...
     3        base.{env}.* config files     config/base.prod.yaml, ...
     4        local.* config files          config/local.yaml, local.json, ...
     5        local.{env}.* config files    config/local.prod.yaml, ...
     6        default / script-name files   config/default.yaml, myapp.yaml, ...
     7        config_file / config_files    config_file => '/etc/myapp.yaml'
     8        Environment variables         APP_DATABASE__HOST=db.prod.example.com
     9 (hi)  CLI arguments (@ARGV)          --APP_DATABASE__HOST=db.prod.example.com

The active environment is set by the C<environment> constructor option, or
auto-detected from C<{env_prefix}ENV> (e.g. C<APP_ENV>), C<PLACK_ENV>, or
C<NODE_ENV>.  When no environment is configured, rows 3 and 5 are skipped.

Within the file tier (rows 2-7), later files override earlier files using
C<Hash::Merge> LEFT_PRECEDENT: every key in the later file wins over the same
key in an earlier file, even when the later value is C<undef> (YAML C<~>).
Nested hashes are merged recursively, so a C<local.yaml> that only sets
C<database.host> will not erase C<database.port> from C<base.yaml>.

  Example - what wins for the key C<database.host> when APP_ENV=prod:

  data              =>  'localhost'              (overridden by base.yaml)
  base.yaml         =>  'db.example.com'         (overridden by base.prod.yaml)
  base.prod.yaml    =>  'db.prod.example.com'    (overridden by local.yaml)
  local.yaml        =>  'db.local.example.com'   (overridden by local.prod.yaml)
  local.prod.yaml   =>  'db.prod-local.example.com'  (overridden by $APP_DATABASE__HOST)
  $APP_DATABASE__HOST  =>  'db.override.example.com'    <-- this wins

=head2 KEY FEATURES

=over 4

=item * Multi-Format Support

Supports configuration files in YAML, JSON, XML, and INI formats.
Automatically merges configuration data from these different formats,
allowing hierarchical configuration management.

=item * Environment Variable Overrides

Allows environment variables to override values in the configuration files.
By setting environment variables with a specific prefix (default: C<APP_>),
values in the configuration files can be dynamically adjusted without modifying
the file contents.

=item * Flattened Configuration Option

Optionally supports flattening the configuration structure. This converts deeply
nested configuration keys into a flat key-value format (e.g., C<database.user>
instead of C<database-E<gt>{user}>). This makes accessing values easier for
applications that prefer flat structures or need compatibility with flat
key-value stores.

=item * Layered Configuration

Supports merging multiple layers of configuration files. For example, you can
have a C<base.yaml> configuration file that provides default values, and a
C<local.yaml> (or C<local.json>, C<local.xml>, etc.) file that overrides
specific values. This allows for environment-specific configurations while
keeping defaults intact.

=item * Merge Strategy

The module merges the configuration data intelligently, allowing values in more
specific files (like C<local.yaml>, C<local.json>, C<local.xml>, C<local.ini>)
to override values in base files. This enables a flexible and layered configuration
system where you can set defaults and override them for specific environments.

=item * Error Handling

Includes error handling for loading configuration files.
If any file fails to
load (e.g., due to syntax issues), the module will throw descriptive error
messages to help with debugging.

=item * Centralized Configuration Management

Supports remote configuration management files,
so that configuration on remote machines can be centrally managed.

=back

=head2 SUPPORTED FILE FORMATS

=over 4

=item * YAML (C<*.yaml>, C<*.yml>)

Loaded with C<YAML::XS>.

=item * JSON (C<*.json>)

Loaded with C<JSON::MaybeXS>.

=item * XML (C<*.xml>)

Loaded with C<XML::Simple> (preferred) or C<XML::PP> (fallback).

=item * INI (C<*.ini>)

Loaded with C<Config::IniFiles>.

=item * TOML (C<*.toml>)

Loaded with C<TOML::Tiny>, which implements TOML 1.0.  TOML is a good choice
for human-editable configuration because its syntax is unambiguous: all values
are typed, strings must be quoted, and nesting uses C<[section]> headers or
dotted keys rather than indentation.

  # Example: config/base.toml
  [database]
  host   = "db.example.com"
  port   = 5432
  user   = "app"

  [cache]
  ttl    = 300
  debug  = false

C<base.toml> and C<local.toml> are discovered automatically in C<config_dirs>;
C<local.toml> has higher precedence than C<base.toml>, following the same
layering rules as all other formats.  TOML is also tried as a fallback parser
in the all-parsers chain used for extensionless C<config_file> entries.

If C<TOML::Tiny> is not installed, TOML files are silently skipped with a
C<carp> warning.

=back

=head2 ENVIRONMENT VARIABLE HANDLING

Configuration values can be overridden via environment variables. Environment variables use double underscores (__) to denote nested configuration keys and single underscores remain as part of the key name under the prefix namespace.

For example:

  APP_DATABASE__USER becomes database.user (nested structure)

    $ export APP_DATABASE__USER="env_user"

will override any value set for `database.user` in the configuration files.

  APP_LOGLEVEL becomes APP.loglevel (flat under prefix namespace)

  APP_API__RATE_LIMIT becomes api.rate_limit (mixed usage)

This allows you to override both top-level and nested configuration values using environment variables.

Configuration values can be overridden via the command line (C<@ARGV>).
For instance, if you have a key in the configuration such as C<database.user>,
you can override it by adding C<"--APP_DATABASE__USER=other_user_name"> to the command line arguments.
This will override any value set for C<database.user> in the configuration files.

=head2 EXAMPLE CONFIGURATION FLOW

=over 4

=item 1. Data Argument

The data passed into the constructor via the C<data> argument is the starting point.
Essentially,
this contains the default values.

=item 2. Loading Files

The module then looks for configuration files in the specified directories.
It loads the following files in order of preference:
C<base.yaml>, C<local.yaml>, C<base.json>, C<local.json>, C<base.xml>,
C<local.xml>, C<base.ini>, and C<local.ini>.

If C<config_file> or C<config_files> is set, those files are loaded last.

If no C<config_dirs> is given, try hard to find the files in various places.

The value of C<config_dirs> can be overridden at runtime by the environment variable CONFIG_DIR
(note that is just one directory, hence it's CONFIG_DIR not CONFIG_DIRS).

=item 3. Merging and Resolving

The module merges the contents of these files, with more specific configurations
(e.g., C<local.*>) overriding general ones (e.g., C<base.*>).

=item 4. Environment Overrides

After loading and merging the configuration files,
the environment variables are
checked and used to override any conflicting settings.

=item 5. Command Line

Next, the command line arguments are checked and used to override any conflicting settings.

=item 6. Accessing Values

Values in the configuration can be accessed using a dotted notation
(e.g., C<'database.user'>), regardless of the file format used.

=back

=head1 METHODS

=head2 new

Constructor for creating a new configuration object.

Options:

=over 4

=item * C<config_dirs>

An arrayref of directories to look for configuration files
(default: C<$CONFIG_DIR>, C<$HOME/.conf>, C<$HOME/config>, C<$HOME/conf>, C<$DOCUMENT_ROOT/conf>, C<$DOCUMENT_ROOT/../conf>, C<conf>).

For centralised configuration management,
entries beginning with C</../> are treated as remote specifications using the
Newcastle Connection convention -- see L</Remote configuration directories (Newcastle Connection)>.

=item * C<config_file>

Points to a configuration file of any format.

=item * C<config_files>

An arrayref of files to look for in the configuration directories.
Put the more important files later,
since later files override earlier ones.

Considers the files C<default> and C<$script_name> before looking at C<config_file> and C<config_files>.

=item * C<data>

A hash ref of default data to prime the configuration with.
These are applied before loading
other sources and can be overridden by later sources or by explicitly passing
options directly to C<new>.

  $config = Config::Abstraction->new(
      data => {
          log_level => 'info',
          retries => 3,
      }
  );

=item * C<defaults>

A hash reference that provides default values for the object's own attributes (such as C<config_dirs>, C<logger>, C<flatten>, etc.).
If this option is supplied,
the object is initialized using the keys in this hash as the base;
any other options passed directly to C<new()> (aside from C<env_prefix>) are ignored.
This allows you to pre-define a standard configuration profile for the object itself.
Note that C<defaults> is distinct from the C<data> option - C<data> supplies the initial configuration values that will be merged with files, environment, and command line,
while C<defaults> sets the object's internal parameters.
The C<env_prefix> value,
if provided as a top-level argument,
still takes precedence over any C<env_prefix> that might exist inside the C<defaults> hash.

=item * C<encryption_key>

A 256-bit (32-byte) AES key used to transparently decrypt C<ENC[...]> values found in any
configuration source after the full merge.  The key may be supplied as:

=over 4

=item 32 raw bytes

=item 64 lowercase or uppercase hex characters

=item 44 Base64 or Base64url characters (standard or URL-safe alphabet, with or without trailing C<=>)

=back

If no key is configured (and neither C<encryption_key_file> nor the corresponding
environment variables are set), C<ENC[...]> tokens are left as literal strings.
Decryption requires L<CryptX> (C<Crypt::AuthEnc::GCM>); if that module is absent a
C<croak> is raised when an encrypted value is encountered.

See L</ENCRYPTED VALUES> for the full workflow.

=item * C<encryption_key_file>

Path to a file whose first line contains the encryption key in any of the formats accepted
by C<encryption_key>.  Takes precedence over the C<ENCRYPTION_KEY_FILE> and
C<{env_prefix}ENCRYPTION_KEY_FILE> environment variables, but is overridden by a key
supplied directly via C<encryption_key>.

=item * C<env_prefix>

A prefix for environment variable keys and comment line options, e.g. C<MYAPP_DATABASE__USER>,
(default: C<'APP_'>).

=item * C<environment>

The name of the active deployment environment (e.g. C<'dev'>, C<'staging'>, C<'prod'>).
When set, config files named C<base.{env}.*> are loaded immediately after their C<base.*>
equivalents, and C<local.{env}.*> files are loaded immediately after C<local.*> files,
giving two additional override tiers at no cost to the base configuration.

  my $cfg = Config::Abstraction->new(
      config_dirs => ['config'],
      environment => 'prod',          # loads base.prod.yaml, local.prod.yaml, ...
  );

If C<environment> is not given, the value is auto-detected in this order:

=over 4

=item 1. C<{env_prefix}ENV> environment variable (e.g. C<APP_ENV>)

=item 2. C<PLACK_ENV>

=item 3. C<NODE_ENV>

=back

When none of these is set, the environment-specific tiers are silently skipped.
The environment name must contain only ASCII letters, digits, hyphens, and underscores
(matched against C</^[A-Za-z0-9_\-]+$/>); any other value causes a C<croak>.

=item * C<file>

Synonym for C<config_file>

=item * C<flatten>

If true, returns a flat hash structure like C<{database.user}> (default: C<0>) instead of C<{database}{user}>.
`
=item * C<level>

Level for logging.

=item * C<logger>

Used for warnings and traces.
It can be an object that understands warn() and trace() messages,
such as a L<Log::Log4perl> or L<Log::Any> object,
a reference to code,
a reference to an array,
or a filename.

=item * C<path>

A synonym of C<config_dirs>.

=item * C<sep_char>

The separator in keys.
The default is a C<'.'>,
as in dotted notation,
such as C<'database.user'>.

=item * C<schema>

A L<Params::Validate::Strict> compatible schema to validate the configuration file against.

=item * C<validators>

A hashref mapping dotted config keys to validation rules.  Each rule is applied to the
corresponding value in the merged configuration immediately after all sources have been
merged (or on first access when C<lazy> is set).  A validation failure croaks.

Each rule may be one of:

=over 4

=item A type name string

  validators => {
      'database.port' => 'integer',
      'app.name'      => 'string',
      'price'         => 'number',
      'enabled'       => 'boolean',
      'tags'          => 'array',
      'settings'      => 'hash',
  }

Supported types: C<integer> (matches C</^-?\d+$/> -- no decimal point),
C<number> or C<float> (C<Scalar::Util::looks_like_number>), C<boolean>
(C<0>, C<1>, C<true>, C<false>, C<yes>, C<no> -- case-insensitive),
C<string> (any defined non-reference scalar), C<array> (arrayref),
C<hash> (hashref).

=item A compiled regular expression

  validators => {
      'log.level' => qr/^(?:debug|info|warn|error|fatal)$/i,
      'app.name'  => qr/^\w[\w\-]{1,63}$/,
  }

The value must be defined and must match the regex.

=item A coderef

  validators => {
      'database.port' => sub { my $v = shift; defined($v) && $v >= 1 && $v <= 65535 },
  }

Called with the value as its only argument.  Must return a true value; otherwise the
constructor croaks.

=item A hashref combining multiple constraints

  validators => {
      'database.port' => {
          type     => 'integer',
          min      => 1,
          max      => 65535,
          required => 1,
      },
      'api.key' => {
          pattern  => qr/^[A-Za-z0-9]{32}$/,
          required => 1,
      },
  }

Keys: C<type> (type-name string as above), C<pattern> (compiled regex), C<min> (numeric
lower bound, inclusive), C<max> (numeric upper bound, inclusive), C<required> (if true,
the key must exist and its value must be defined).

=back

All constraint types may be freely combined: specifying both C<type> and C<pattern>
requires the value to satisfy both.

=item * C<checker>

A prototype string (YAML) or hashref passed to L<Config::Checker> for template-based
structural validation.  C<Config::Checker> must be installed; if it is absent a C<carp>
warning is emitted and validation is skipped.

The prototype mirrors the expected config structure.  Keys and values can carry type
annotations (C<[INTEGER]>, C<[PATH]>, C<[HOSTNAME]>, ...), custom code checks
(C<{...}>), and quantity specifiers (C<?>: optional, C<+>: one or more, C<*>: zero or
more):

  my $config = Config::Abstraction->new(
      config_dirs => ['config'],
      checker     => <<'END_PROTOTYPE',
  database:
    host: hostname of the database server[HOSTNAME]
    port: '?<5432>port number[INTEGER]'
    user: database username
  END_PROTOTYPE
  );

See L<Config::Checker> for the full prototype syntax.

=item * C<lazy>

When set to a true value, all source discovery and file I/O are deferred until the first
call to C<get()>, C<exists()>, C<all()>, C<explain_sources()>, C<prefer_*()>,
C<merge_defaults()>, or any AUTOLOAD accessor.  This can reduce startup time in
applications that construct a C<Config::Abstraction> object before they know whether
they will access it.

B<Differences from the default (eager) behaviour, and the debugging problems they cause:>

=over 4

=item 1. C<new()> always succeeds, even when no configuration exists.

In eager mode, C<new()> returns C<undef> immediately when no files are found and no
C<data> was supplied, so a missing config is caught at the point of construction.
In lazy mode, C<new()> always returns a blessed object.  If the configuration
directories do not exist, the files are missing, or the prefix is wrong, you will
not find out until the first accessor call - which may be deep inside your business
logic, far from where the object was created.

  # Problem: the typo in config_dirs goes unnoticed until runtime
  my $cfg = Config::Abstraction->new(
      config_dirs => ['/etc/myapp/conifg'],   # typo - directory does not exist
      lazy => 1,
  );
  # ... many lines later ...
  my $host = $cfg->get('database.host');      # silently returns undef here

=item 2. Schema validation errors appear at the wrong place in the call stack.

When a C<schema> is supplied, eager mode validates the merged configuration inside
C<new()> and throws immediately if the config does not match.  In lazy mode,
validation is deferred to the first accessor call.  A schema violation therefore
surfaces as an exception thrown by C<get()> or C<all()>, not by C<new()>, and the
stack trace points to the accessor call rather than the construction site.  If the
object is created in one module and accessed in another, the error message can be
very misleading.

=item 3. Environment variables and C<@ARGV> are captured at access time, not construction time.

The module reads C<%ENV> and C<@ARGV> during C<_load_config()>.  In eager mode that
happens in C<new()>, so the configuration reflects the environment at the moment the
object is built.  In lazy mode, any change to C<%ENV> or C<@ARGV> between C<new()>
and the first accessor call is silently picked up.  This makes test isolation harder:
setting an environment variable I<after> constructing a lazy object will affect the
values it returns, which is not the case with an eager object.

=item 4. File system state may change between construction and first use.

Because lazy mode defers all I/O, the files that are read are those that exist at the
moment of first access, not at the moment C<new()> is called.  If a config file is
written, deleted, or replaced between those two points (for example, by another
process or test fixture), the object will silently use the new state.  An eager object
is immune to this race.

=item 5. Errors from format parsers appear at accessor call sites.

Malformed YAML, JSON, XML, or INI files emit a C<carp> warning and are skipped during
loading.  In eager mode those warnings appear near the program's startup.  In lazy mode
they appear during the first accessor call, which can make it look as though a routine
in your business logic is generating configuration warnings.

=back

B<When to use lazy loading:> It is most useful when a C<Config::Abstraction> object is
created speculatively at module-load time and may never be accessed (for example, in a
web framework where the config object is built for every request but some request paths
never read it).  Avoid it when startup correctness is important, when you rely on
C<new()> returning C<undef> to detect missing configuration, or when you use schema
validation and need errors to point at the construction site.

=back

If just one argument is given, it is assumed to be the name of a file.

=cut