File Coverage

=head1 NAME

Geo::Coder::Free - Geocoding using free, locally-hosted databases

=head1 VERSION

Version 0.43

=cut

=head1 SYNOPSIS

    use Geo::Coder::Free;

    my $geo = Geo::Coder::Free->new();
    my $pt  = $geo->geocode(location => 'Ramsgate, Kent, UK');
    printf "%.6f, %.6f\n", $pt->lat(), $pt->long();

    # With OpenAddresses/WhoOnFirst data:
    my $geo2 = Geo::Coder::Free->new(openaddr => $ENV{OPENADDR_HOME});
    my $pt2  = $geo2->geocode(location => '1600 Pennsylvania Avenue NW, Washington DC, USA');

    # Free-text scanning:
    my @hits = $geo2->geocode(scantext => 'She grew up in Ramsgate, Kent.',
                              region   => 'GB');

=head1 DESCRIPTION

C<Geo::Coder::Free> translates addresses into latitude/longitude coordinates
using local SQLite databases built from free data sources - MaxMind/GeoNames,
OpenAddresses, Who's On First, OpenStreetMap, and dr5hn's countries/states/cities
database.  It deliberately avoids paid or rate-limited online geocoding services.
The module is designed to be flexible, supporting both command-line and programmatic usage.
It also includes a sample CGI script for a web-based geocoding service.

Geocoding dispatch order depends on whether C<OPENADDR_HOME> (or C<openaddr>) is set:

B<With OpenAddresses data:>

=over 4

=item 1. C<Geo::Coder::Free::OpenAddresses> - requires C<OPENADDR_HOME>

=item 2. C<Geo::Coder::Free::Local> - user-curated CSV entries (tried as fallback)

=item 3. C<Geo::Coder::Free::MaxMind> - bundled, always available

=back

B<Without OpenAddresses data:>

=over 4

=item 1. C<Geo::Coder::Free::MaxMind> only - Local is not consulted.

=back

The C<cgi-bin> directory contains a simple DIY geo-coding website:

    cgi-bin/page.fcgi page=query q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA

The sample website is currently down while a new host is sought.
When it returns, you will be able to test it with:

    curl 'https://geocode.nigelhorne.com/cgi-bin/page.fcgi?page=query&q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA'

=head1 LIMITATIONS

=over 4

=item * C<scantext> mode only finds locations in OpenAddresses; it falls back
silently when C<OPENADDR_HOME> is not set (B<FIXME>: should warn).

=item * The C<__DATA__> alternatives table is hard-coded; it should live in a
user-editable config file.

=item * The address-regex scantext path misses birth-year sentences such as
C<"She was born May 21, 1937 in Noblesville, IN."> because the regex requires
a preceding capital-letter word directly before the city.

=item * C<reverse_geocode> is only partially implemented; the MaxMind path does
not return meaningful results.

=item * The C<alternatives> map loop uses C<each %{$alt}>, which retains its
iterator position across calls.  After a successful match and early C<return>,
the next C<geocode()> call on the same input starts iterating from the key
B<after> the matched one, potentially missing the match entirely until C<each>
wraps around.  Workaround: call C<keys %{$alt}> once to reset the iterator
before iterating.

=back

=cut

=head1 METHODS

=head2 new

=head3 SYNOPSIS

    my $geo = Geo::Coder::Free->new();
    my $geo = Geo::Coder::Free->new(openaddr => '/data/openaddr');
    my $geo = Geo::Coder::Free->new(directory => '/data/maxmind');

=head3 DESCRIPTION

Constructor.  Accepts a hash or hashref of options.  If called without
C<openaddr>, the module checks C<$ENV{OPENADDR_HOME}> before giving up.

If called on an existing object instance (C<< $clone = $geo->new() >>), returns
a B<shallow clone>.  All scalar fields are copied by value, but reference-type
fields (C<alternatives>, C<scantext_misses>, C<maxmind>, C<openaddr>) share the
same underlying object or hashref between the original and the clone.  Mutations
to those shared references are immediately visible in both objects.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    openaddr  => { type => 'scalar', optional => 1 }                          # path to OpenAddresses/WOF data dir
    directory => { type => 'scalar', optional => 1 }                          # path to MaxMind/GeoNames files
    cache     => { type => 'object', optional => 1, can => ['get', 'set'] }   # CHI-compatible cache object

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Coder::Free' }

=head3 EXAMPLE

    use Geo::Coder::Free;

    # Minimal - uses only the bundled MaxMind data:
    my $geo = Geo::Coder::Free->new();

    # Full - also searches OpenAddresses/WOF:
    my $geo = Geo::Coder::Free->new(openaddr => $ENV{OPENADDR_HOME});

=head3 MESSAGES

    use ->new() not ::new()   Called as a function; use arrow syntax.

=cut

=head2 geocode

=head3 SYNOPSIS

    # Standard lookup (returns a Geo::Location::Point or undef)
    my $pt = $geo->geocode(location => 'Ramsgate, Kent, UK');
    printf "lat=%.6f lon=%.6f\n", $pt->lat(), $pt->long();

    # Scantext - returns a list of Geo::Location::Point objects
    my @hits = $geo->geocode(
        scantext     => 'She lived in Ramsgate, Kent.',
        region       => 'GB',
        ignore_words => [qw(lived)],
    );

    # Invocation flexibility (all equivalent)
    $geo->geocode('Ramsgate, Kent, UK');
    $geo->geocode({ location => 'Ramsgate, Kent, UK' });
    $geo->geocode(location => 'Ramsgate, Kent, UK');

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) - exactly one of location or scantext is required
    location     => { type => 'scalar',   optional => 1 }  # address string (exclusive with scantext)
    scantext     => { type => 'scalar',   optional => 1 }  # free text to scan for place names
    region       => { type => 'scalar',   optional => 1 }  # ISO 3166-1 alpha-2 country code hint
    ignore_words => { type => 'arrayref', optional => 1 }  # words to suppress during scantext scan

=head4 output

    # Output schema (Return::Set)
    # scalar context: { type => 'object',   isa => 'Geo::Location::Point', optional => 1 }
    # list context:   { type => 'arrayref', of  => { isa => 'Geo::Location::Point' } }

=head3 MESSAGES

    Usage: ...::geocode(...)        No location or scantext argument given.
    invalid location to geocode()   location is purely numeric.
    invalid scantext to geocode()   scantext is purely numeric.

=head3 PSEUDOCODE

    if self is not a blessed object → delegate to new()->geocode(@args)
    normalise @_ into %params
    validate: location is not purely numeric; scantext is not purely numeric
    if openaddr backend is available:
        if scantext:
            try the raw scantext string as a direct location
            build stopword set from %_COMMON_WORDS + ignore_words param
            try 3-word windows (triplets) at confidence 0.8
            try 2-word windows (duplets) at confidence 0.7
            try the address-pattern regex at confidence 0.7
            try region-specific address finders (GB / US / CA)
            mark scantext as a miss; return undef
        else:
            try openaddr backend
            try local backend
            try __DATA__ alternatives map
    try maxmind backend for location lookups
    croak if no scantext and no location

=cut

=head2 reverse_geocode

=head3 SYNOPSIS

    my $loc = $geo->reverse_geocode(latlng => '51.3341,-1.4159');

=head3 DESCRIPTION

Translates a latitude/longitude pair back to a place name.
B<Partially implemented>: the MaxMind backend does not return meaningful results.
OpenAddresses is attempted first when available.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) — latlng required
    latlng => { type => 'scalar' }  # "$lat,$long" comma-separated decimal degrees
    # NOTE: separate lat/lon/long keys are NOT supported at the Geo::Coder::Free
    # (facade) level.  When no OpenAddresses backend is configured, passing
    # lat/lon/long instead of latlng will croak "not yet supported".
    # To use separate coordinates call Geo::Coder::Free::Local::reverse_geocode.

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Location::Point', optional => 1 }

=cut

=head2 ua

Does nothing.  Present for drop-in compatibility with other Geo::Coder::* modules.

=cut

=head2 run

Command-line entry point.  Use as:

    perl lib/Geo/Coder/Free.pm 1600 Pennsylvania Avenue NW, Washington DC

=cut

=head1 GETTING STARTED

To download, import and set up the local database:
before running C<make>, but after running C<perl Makefile.PL>, follow these instructions.

Optionally set C<OPENADDR_HOME> to point to an empty directory and download the data from
L<http://results.openaddresses.io> into that directory; and
optionally set C<WHOSONFIRST_HOME> to point to an empty directory and download the data using
L<https://github.com/nigelhorne/NJH-Snippets/blob/master/bin/wof-clone>.
The script C<bin/download_databases> (see below) will do those for you.
You do not need to download the MaxMind data — that is downloaded automatically.

You will need to create the database used by C<Geo::Coder::Free>.

Install L<App::csv2sqlite> and L<https://github.com/nigelhorne/NJH-Snippets>.
Run C<bin/create_sqlite> — this converts the MaxMind "cities" database from CSV to SQLite.

To use with MariaDB, set C<MARIADB_SERVER="$hostname;$port"> and
C<MARIADB_USER="$user;$password"> (TODO: username/password should be asked for interactively).
The code will use a database called C<geo_code_free>, which will be dropped and recreated if it exists.
C<$user> needs only DROP, CREATE, SELECT, INSERT, and INDEX privileges on that database.

The following optional steps download and install large databases.
This will take a long time and use a lot of disc space.

=over 4

=item 1

C<mkdir $WHOSONFIRST_HOME; cd $WHOSONFIRST_HOME> then run C<wof-clone> from NJH-Snippets.

This can take a long time because it contains many nested directories, which filesystem drivers
can be slow to navigate (particularly on EXT4 and ZFS).

=item 2

Install L<https://github.com/dr5hn/countries-states-cities-database.git> into C<$DR5HN_HOME>.
This data covers cities only, so it is not used when C<OSM_HOME> is set (OSM is far more
comprehensive).  Only Australia, Canada, and the US are imported, as the UK data is difficult
to parse.

=item 3

Run C<bin/download_databases> — this downloads the Who's On First, OpenAddr, OpenStreetMap,
and dr5hn databases.
OpenStreetMap now uses PBF files, so you will need C<apt install osmium-tool> first.
Check the values of C<OSM_HOME>, C<OPENADDR_HOME>, C<DR5HN_HOME> and C<WHOSONFIRST_HOME>
within that script and adjust them for your setup.
The C<Makefile.PL> file downloads the MaxMind database automatically, as it is not optional.

=item 4

Run C<bin/create_db> — this creates the database used by C<Geo::Coder::Free> from the data you
have just downloaded.
The database is called C<openaddr.sql> for historical reasons (before Who's On First was added);
it actually contains data from all sources above.

=back

Now you are ready to run C<make>.
See the comment at the start of C<createdatabase.PL> for further details.

=head1 MORE INFORMATION

I have written several Perl genealogy programs including
L<gedcom|https://github.com/nigelhorne/gedcom> and
L<ged2site|https://github.com/nigelhorne/ged2site>.
One of the things these do is check the validity of a family tree, including verifying place-names.
Of course places do change names and spelling becomes more consistent over the years, but the vast
majority remain the same — enough to make computerised verification worthwhile.

=head1 BUGS

Some lookups fail.  Please file a bug report at
L<https://rt.cpan.org/NoAuth/Bugs.html?Dist=Geo-Coder-Free>.

The MaxMind data contains cities only.
The OpenAddresses data does not cover the whole globe.
C<London, England> cannot be parsed yet.

=head1 SEE ALSO

=over 4

=item * L<Configure an Object at Runtime|Object::Configure>

=item * L<Test Dashboard|https://nigelhorne.github.io/Geo-Coder-Free/coverage/>

=back

L<Geo::Coder::Free::Local>, L<Geo::Coder::Free::MaxMind>,
L<Geo::Coder::Free::OpenAddresses>,
L<https://openaddresses.io/>, L<https://www.maxmind.com/>,
L<https://www.geonames.org/>, L<https://www.whosonfirst.org/>.

=head1 AUTHOR

Nigel Horne C<< <njh@nigelhorne.com> >>

=head1 FORMAL SPECIFICATION

=head2 new

    GeoCoderFreeState ::= ⟨⟨ maxmind     : MaxMind_Geocoder;
                              openaddr    : OpenAddr_Geocoder | undef;
                              alternatives: Map[STRING → STRING];
                              cache       : Cache | undef ⟩⟩

    Init : Params → GeoCoderFreeState
    âˆ€ p : Params •
      let oa_path == p.openaddr ∨ env.OPENADDR_HOME •
      GeoCoderFreeState.openaddr = if oa_path ≠ ∅ then OpenAddresses(oa_path) else undef fi

=head2 geocode

    Geocode : Address × Region? → Point?
    âˆ€ addr : Address; r : Region? •
      let backends == (openaddr ≠ undef ⟹ [OpenAddresses, Local, MaxMind])
                    âˆ§ (openaddr = undef ⟹ [MaxMind]) •
      result = first { defined } map { b.geocode(addr, r) } backends

=head1 LICENSE AND COPYRIGHT

Copyright 2017-2026 Nigel Horne.  Licensed under GPL2 for personal use.

This product uses GeoLite2 data created by MaxMind,
available from L<https://www.maxmind.com/>.

=cut