File Coverage

=head1 NAME

Geo::Coder::List - Call many Geo-Coders

=head1 VERSION

Version 0.39

=cut

=head1 SYNOPSIS

    use Geo::Coder::List;
    use Geo::Coder::OSM;

    # 1. The simplest use: one list, one geocoder
    my $list = Geo::Coder::List->new()->push(Geo::Coder::OSM->new());

    my $place = $list->geocode('10 Downing St, London, UK');
    if($place) {
        print "Latitude:  $place->{geometry}{location}{lat}\n";
        print "Longitude: $place->{geometry}{location}{lng}\n";
    } else {
        print "Not found\n";
    }

    # 2. Several geocoders, tried in order.  Geo::Coder::CA is asked only
    #    about places in Canada or the USA, and at most 1000 times.
    use Geo::Coder::CA;

    my $chain = Geo::Coder::List->new()
        ->push({ regex => qr/(Canada|USA)$/, geocoder => Geo::Coder::CA->new(), limit => 1000 })
        ->push(Geo::Coder::OSM->new());

    my $ottawa = $chain->geocode('Ottawa, ON, Canada');    # asks CA first
    my $paris  = $chain->geocode('Paris, France');         # CA is skipped

    # 3. A cache that lives longer than the object (any CHI driver works)
    use CHI;

    my $cached = Geo::Coder::List->new(
        cache => CHI->new(driver => 'File', root_dir => '/var/cache/geocode'),
    )->push(Geo::Coder::OSM->new());

    # 4. Every candidate, not only the best one: call geocode() in list context
    my @candidates = $list->geocode('Springfield, USA');
    print scalar(@candidates), " places are called Springfield\n";

    # 5. From a position to an address
    my $address = $list->reverse_geocode(latlng => '51.5034,-0.1276');
    print "$address\n" if defined($address);

    # 6. Bad input is fatal, so check it when the text comes from a user
    my $text_from_a_form = $ARGV[0];
    my $result = eval { $list->geocode($text_from_a_form) };
    warn "Not a place: $@" if $@;

    # 7. See what happened, then clear the record
    $list->geocode('Rome, Italy');
    for my $entry (@{ $list->log() }) {
        printf "%s: %s in %.3fs\n", $entry->{location}, $entry->{geocoder}, $entry->{timetaken};
    }
    $list->flush();

=head1 DESCRIPTION

L<Geo::Coder::All> and L<Geo::Coder::Many> are great modules but neither
quite does what I want.

A I<geocoder> turns the name of a place, such as C<'Paris, France'>, into
a position on the Earth (a latitude and a longitude).  There are many
geocoding services.  Each one knows some places better than others, each
one has its own limits, and each one returns its answer in its own format.

C<Geo::Coder::List> lets you use many geocoders as if they were one:

=over 4

=item * B<A chain.>
You add geocoders with C<push()>.  When you ask for a place, the geocoders
are asked one at a time, in the order you added them.  The first one that
gives a usable answer wins, and the others are not asked.

=item * B<Routing.>
A geocoder can be given a regular expression.  It is then asked only
about places whose names match it.

=item * B<Limits.>
A geocoder can be given a limit: the number of questions it may answer.
When the limit is used up, the geocoder is skipped.

=item * B<One answer format.>
Every answer is changed into the same structure, whichever service gave
it.  The structure is the one used by L<HTML::GoogleMaps::V3> and
L<HTML::OSM>:

    $result->{geometry}{location}{lat}   # latitude
    $result->{geometry}{location}{lng}   # longitude
    $result->{lat}                       # the same latitude, for convenience
    $result->{lng}                       # the same longitude, for convenience
    $result->{lon}                       # the same longitude, another common name
    $result->{geocoder}                  # the geocoder object that answered,
                                         # or the string 'cache'

=item * B<A cache in two levels.>
Level 1 (L1) is a hash inside the object.  It is always used, and it lasts
as long as the object.  Level 2 (L2) is optional.  It can be a plain hash
or any object with C<get()> and C<set()> methods, such as a L<CHI> cache.
L2 can last longer than the object and can be shared between programs.
How long L2 keeps an answer depends on the answer:

    found                      1 month   (cache_hit_duration)
    found, but no latitude     1 day     (cache_part_duration)
    not found                  1 week    (cache_miss_duration)

A place that was not found is remembered in L1 only, never in L2.

=back

=head1 SUPPORTED ANSWER FORMATS

Each geocoding service answers in its own format.  C<geocode()> finds the
latitude and longitude in each of the formats below and copies them into
the common structure.  Any geocoder whose answers use one of these formats
works without extra code.  The provider's own fields are kept as well.

    Service (module)                Where the position is found
    ------------------------------  -----------------------------------------
    OpenStreetMap, RandMcNally      lat, lon
    Bing Maps                       BestLocation.Coordinates.Latitude/Longitude
    Bing Maps (other form)          point.coordinates [lat, lng]
    geocoder.ca (Geo::Coder::CA)    latt, longt
    postcodes.io, Geo::Coder::Free  latitude, longitude
    HERE / Ovi                      properties.geoLatitude/geoLongitude
    DataScienceToolkit              results[0].geometry.location.lat/lng
    OpenCage                        results[0].geometry.lat/lng
    GeocodeFarm                     RESULTS[0].COORDINATES.latitude/longitude
    US Census Bureau                result.addressMatches[0].coordinates.y/x
    GeoNames                        lat, lng
    Mapbox                          features[0].center [lng, lat]
    GeoApify                        features[0].geometry.coordinates [lng, lat]
    Google, and anything already    geometry.location.lat/lng
    in the common structure

Notice that Mapbox and GeoApify give the longitude B<first>.

Some services need special handling, which is done for you:

=over 4

=item * B<Geo::Location::Point.>
A geocoder may answer with a L<Geo::Location::Point> object.  Its
C<geometry> is filled in from its own C<lat> and C<lng> (or C<lon>).

=item * B<postcodes.io.>
Its C<local_type> field is copied to C<type>, with the first letter made
lower case.

=item * B<geocoder.xyz.>
Its C<standard.countryname> field is copied to C<address.country>.

=item * B<GeoNames.>
L<Geo::GeoNames> is called with the place name alone, not with
C<location =E<gt> ...>.  If it has no C<username>, it fails with
C<lost username>, which is logged, and the next geocoder is asked.
GeoNames puts its answers in arrays; every answer in them is considered.

=item * B<US Census Bureau.>
An answer with no C<addressMatches> means "not found", and the next
geocoder is asked.

=item * B<GeoApify.>
An answer with an empty C<features> list means "not found", and the next
geocoder is asked.

=item * B<Errors.>
An answer with a top-level C<error> key is treated as a refusal, not as a
place.  The error is logged, and the next geocoder is asked.

=back

Every answer that went through this process also has a C<debug> key.  It
holds the line number, in this module, of the code that read the answer.
It is there to help when a new format is not read correctly.

=head1 SUBROUTINES/METHODS

=head2 new

Creates a new C<Geo::Coder::List> object with no geocoders in it.

Settings can also come from the environment, through
L<Object::Configure>.  The variable name is C<GEO__CODER__LIST__> followed
by the setting name.  For example, C<GEO__CODER__LIST__carp_on_warn=1>
makes warnings go through L<Carp>, and
C<GEO__CODER__LIST__cache_hit_duration='7 days'> changes how long a found
place stays in L2.

When you call C<new()> on an object that already exists, you get a
I<clone>: a new object with the same settings and the same geocoders,
merged with any arguments you give.  The clone has its own list of
geocoders and its own limits, and it starts with an empty log.  The clone
B<shares> the L1 and L2 caches with the original.

    use Geo::Coder::List;
    use CHI;

    # With an L2 cache
    my $geocoder = Geo::Coder::List->new(
        cache => CHI->new(driver => 'Memory', global => 1),
    );

    # A clone of it that prints what it is doing
    my $verbose = $geocoder->new(debug => 1);

Arguments can be given as a list of pairs or as one hash reference.

=over 4

=item * C<cache> - Optional.  The L2 cache: a hash reference, or an object
with C<get($key)> and C<set($key, $value, $ttl)> methods.

=item * C<debug> - Optional.  C<0> (the default) prints nothing, C<1>
prints what is tried, C<2> also prints every answer in full.

=item * C<cache_hit_duration>, C<cache_part_duration>,
C<cache_miss_duration> - Optional.  How long L2 keeps each kind of
answer, in any form your L2 cache understands, such as C<'1 week'>.

=item * C<geocoders> - Optional.  An array reference of geocoders, as if
each had been given to C<push()>.

=item * C<logger> - Optional.  Where warnings and errors go.  See
L<Object::Configure>.

=item * C<carp_on_warn> - Optional.  When true, warnings go through
L<Carp>.

=back

If C<new> is called as a function, C<Geo::Coder::List::new()>, with no
arguments, it still returns an object.  Called as a function with
arguments, it warns and returns C<undef>.

=head3 API SPECIFICATION

=head4 INPUT

    {
        cache               => { type => 'hashref|object', optional => 1 },
        debug               => { type => 'integer', optional => 1, min => 0, max => 2 },
        cache_hit_duration  => { type => 'string', optional => 1 },
        cache_part_duration => { type => 'string', optional => 1 },
        cache_miss_duration => { type => 'string', optional => 1 },
        geocoders           => { type => 'arrayref', optional => 1 },
        logger              => { type => 'object|coderef|arrayref|string', optional => 1 },
        carp_on_warn        => { type => 'boolean', optional => 1 },
    }

=head4 OUTPUT

    {
        type => 'object',
        isa  => 'Geo::Coder::List',
    }

    # undef only when called as Geo::Coder::List::new() with arguments

=cut

=head2 push

Adds one geocoder to the end of the chain.  Geocoders are asked in the
order in which they were added.

C<push> returns the object itself, so you can add several geocoders in
one statement.

There are two ways to add a geocoder:

=over 4

=item * B<A geocoder object.>
It is asked about every place.

=item * B<A hash reference>, with these keys:

=over 4

=item * C<geocoder> - Required.  The geocoder object.

=item * C<regex> - Optional.  A regular expression, made with C<qr//>.
The geocoder is asked only about places whose names match it.

=item * C<limit> - Optional.  A whole number.  The geocoder answers at
most this many questions.  Each question it is asked makes the number one
smaller.  When the number reaches C<0>, the geocoder is skipped.  A place
that does not match C<regex> does not use up the limit.

=back

=back

    my $list = Geo::Coder::List->new()
        ->push({ regex => qr/USA$/, geocoder => Geo::Coder::US::Census->new(), limit => 100 })
        ->push(Geo::Coder::OSM->new());

C<push> croaks if C<$geocoder> is missing or C<undef>.

=head3 API SPECIFICATION

=head4 INPUT

    {
        geocoder => { type => 'object|hashref', optional => 0 },
    }

    # A hashref geocoder holds the keys geocoder (an object, required),
    # regex (a Regexp, optional) and limit (an integer, optional)

=head4 OUTPUT

    {
        type => 'object',
        isa  => 'Geo::Coder::List',
    }

=cut

=head2 geocode

Finds the position of a place from its name.

    my $result = $list->geocode(location => 'Paris, France');
    my $result = $list->geocode('Paris, France');            # the same
    my $result = $list->geocode({ location => 'Paris, France' });  # the same

    if($result) {
        printf "lat=%.4f lng=%.4f from %s\n",
            $result->{geometry}{location}{lat},
            $result->{geometry}{location}{lng},
            ref($result->{geocoder}) || $result->{geocoder};
    }

These are the steps, in order:

=over 4

=item 1. If C<location> is missing, C<geocode> croaks.  If it is
C<undef> or an empty string, C<geocode> warns and returns nothing.

=item 2. Runs of white space are made into one space, and HTML entities
are decoded: C<'Caf&eacute;'> becomes C<'Caf'> followed by the single
character e-acute.  A reference to a string is treated as the string.

=item 3. The text is checked.  A reference is never a place.  When
L<Geo::Location::Predicate> is installed, it decides what else is not a
place; for example, a string of only digits is not.  If the text is not a
place, C<geocode> croaks.  The check is done after step 2, so
C<'&#49;&#50;'> is refused as the digits C<'12'>.

=item 4. The cache is checked: first L1, then L2.  If the place is there,
the cached answer is returned, with C<geocoder> set to C<'cache'>.  If L1
remembers that the place was not found, nothing is returned and no
geocoder is asked.

=item 5. Each geocoder in the chain is asked in turn.  A geocoder is
skipped if the place does not match its C<regex>, or if its C<limit> is
used up.  If a geocoder dies, the error is logged, a warning is given, and
the next geocoder is asked.

=item 6. The first answer that has both a latitude and a longitude wins.
It is changed into the common structure (see L</DESCRIPTION>), stored in
the cache, and returned.

=item 7. If no geocoder gives a usable answer, the place is remembered as
"not found" in L1, and nothing is returned.

=back

B<Scalar context> returns one hash reference, or C<undef>.

B<List context> returns every answer from the geocoder that won, or an
empty list.  This is useful when a name, such as C<'Springfield'>, belongs
to many places.  Answers that the geocoder returned inside an array are
taken out of it, so the list never contains array references.  Answers
without a position may be in the list too; check C<geometry> before you
use each one.

    my @all = $list->geocode('Springfield, USA');

The C<geocoder> key of an answer holds the geocoder object that gave it.
When the answer came from the cache, it holds the string C<'cache'>.

=head3 API SPECIFICATION

=head4 INPUT

    {
        location => { type => 'string', optional => 0, min => 1 },
    }

=head4 OUTPUT

    {
        type => 'hashref',
        optional => 1,
    }

    # Return::Set schema, scalar context: a hashref, or undef when not found
    #   {
    #       geometry => { location => { lat => Num, lng => Num } },
    #       lat      => Num,
    #       lng      => Num,
    #       lon      => Num,                   # the same as lng
    #       geocoder => OBJECT | 'cache',
    #       debug    => Int,                   # line that read the answer
    #       ...                                # fields from the provider
    #   }
    #
    # List context: a list of hashrefs as above, or an empty list

=cut

=head2 ua

Gives a user agent (the object that makes the web requests) to every
geocoder in the chain.  Use it to set a proxy or a time-out for all of
them at once.

    use LWP::UserAgent;

    my $ua = LWP::UserAgent->new(timeout => 10);
    $ua->env_proxy(1);
    $list->ua($ua);

If C<$ua> has both C<clone()> and C<agent()> methods, as
L<LWP::UserAgent> does, each geocoder is given its own copy.  The
C<agent> string of each copy is set to the geocoder's class name and
version, such as C<'Geo::Coder::OSM/0.03'>, so each service can see which
module is calling it.  Otherwise every geocoder is given C<$ua> itself.

Add the geocoders before you call C<ua>: geocoders added later do not get
the user agent.

There is no way to read the user agent back, because each geocoder may
have a different one.

If C<$ua> is missing or false, nothing changes and nothing is returned.
C<ua> croaks with C<No geocoder found> if an entry in the chain has no
geocoder.

=head3 API SPECIFICATION

=head4 INPUT

    {
        ua => { type => 'object', optional => 1 },
    }

=head4 OUTPUT

    {
        type => 'object',
        optional => 1,
    }

    # The same $ua that was passed in, or nothing when $ua is false

=cut

=head2 reverse_geocode

Finds the address of a position: the opposite of C<geocode>.

    my $address = $list->reverse_geocode(latlng => '51.5034,-0.1276');
    my $address = $list->reverse_geocode('51.5034,-0.1276');    # the same
    print "Address: $address\n" if defined($address);

    my @addresses = $list->reverse_geocode(latlng => '51.5034,-0.1276');

C<latlng> is the latitude, a comma, and the longitude, in that order.  It
must be two plain decimal numbers written with the digits C<0> to C<9>.
Spaces, tabs and a C<+> or C<-> sign are allowed.  Anything else, such as
C<'51.5;-0.1'>, C<'1e5,2'> or C<'51.5'>, makes C<reverse_geocode> croak
before any geocoder is asked.

The geocoders are asked in turn, as for C<geocode>.  C<limit> is honoured;
C<regex> is not.  If a geocoder complains that it does not know the
C<latlng> parameter, it is asked again with only C<lat> and C<lon>.  If a
geocoder dies for any other reason, a warning is given and the next
geocoder is asked.

B<Scalar context> returns one address string, or C<undef>.

B<List context> returns every address from the first geocoder that has
any, or an empty list.

An address is taken from the first of these that the answer has:

=over 4

=item * C<display_name> (OpenStreetMap, and others that copy it)

=item * the fields of geocoder.ca: C<stnumber>, C<staddress>, C<city> and
C<prov> for Canada, giving for example
C<'1 Sussex Dr, Ottawa, ON'>; or the C<usa> fields for the United States,
giving for example C<'1600 Pennsylvania Ave, Washington, DC, USA'>

=item * C<features[0].properties.formatted> (GeoApify); only the first
feature is used

=back

If a geocoder returns a plain string, the string is the address.

Addresses are cached under the C<latlng> string, so asking again for the
same position does not ask any geocoder.

=head3 API SPECIFICATION

=head4 INPUT

    {
        latlng => { type => 'string', optional => 0, matches => qr/^[ \t]*[-+]?(?:[0-9]*\.?[0-9]+|[0-9]+\.?[0-9]*)[ \t]*,[ \t]*[-+]?(?:[0-9]*\.?[0-9]+|[0-9]+\.?[0-9]*)[ \t]*$/ },
    }

=head4 OUTPUT

    {
        type => 'string',
        optional => 1,
    }

    # Scalar context: an address string, or undef when not found
    # List context: a list of address strings, or an empty list

=cut

=head2 log

Returns the record of what has happened since the object was made, or
since the last C<flush()>.  There is one entry for each question asked of
a geocoder, and one for each answer that C<geocode()> serves from the
cache.  C<reverse_geocode()> adds no entry when it answers from the
cache.

    for my $entry (@{ $list->log() }) {
        printf "%s: %.3fs via %s\n",
            $entry->{location},
            $entry->{timetaken},
            $entry->{geocoder};
    }

Each entry is a hash reference with these keys:

=over 4

=item * C<location> - the place name or C<latlng> that was asked about

=item * C<geocoder> - the class name of the geocoder, or C<'cache'>

=item * C<timetaken> - how long the geocoder took, in seconds

=item * C<line> - the line of your program that made the call

=item * C<wantarray> - true if the call was made in list context

=item * C<result> - the answer, when there was one; the string
C<'not found'> when the geocoder had none

=item * C<error> - the error, when the geocoder died or returned an
error; this can be an exception object, not only a string

=back

The returned array reference is the object's own record, not a copy.

=head3 API SPECIFICATION

=head4 INPUT

    []

=head4 OUTPUT

    {
        type => 'arrayref',
    }

    # Each element:
    #   {
    #       line      => Int,
    #       location  => Str,
    #       timetaken => Num,
    #       geocoder  => Str,                     # class name, or 'cache'
    #       wantarray => Bool,
    #       result    => HASHREF | ARRAYREF | Str,  # on success
    #       error     => Str | OBJECT | HASHREF,    # on failure
    #   }

=cut

=head2 flush

Empties the log and returns the object itself, so that it can be chained.

    $list->geocode('Paris, France');
    my $entries = $list->log();
    $list->flush()->geocode('London, UK');

C<flush> does B<not> empty the cache.

=head3 API SPECIFICATION

=head4 INPUT

    []

=head4 OUTPUT

    {
        type => 'object',
        isa  => 'Geo::Coder::List',
    }

=cut