File Coverage

File:blib/lib/Geo/Coder/Free/MaxMind.pm
Coverage:37.7%

linestmtbrancondsubpodtimecode
1package Geo::Coder::Free::MaxMind;
2
3# sqlite3 cities.sql
4#       select * from cities where City like '%north shields%';
5# - note 'J5'
6# grep 'GB.ENG.J5' admin2.db
7
8# FIXME: If you search for something like "Sheppy, Kent, England" in list
9#       context, it returns them all.  That's a lot! Should limit to, say
10#       10 results (that number should be tuneable, and be a LIMIT in DB.pm)
11#       And as the correct spelling in Sheppey, arguably it should return nothing
12
13
16
16
16
35
10
220
use strict;
14
16
16
16
25
10
302
use warnings;
15
16
16
16
23
12
35
use autodie qw(:all);
16
17
16
16
16
36150
12
192
use Geo::Coder::Free::MissCache;
18
16
16
16
2257
18
72
use Geo::Coder::Free::DB::MaxMind::admin1;
19
16
16
16
125621
21
68
use Geo::Coder::Free::DB::MaxMind::admin2;
20
16
16
16
123941
22
54
use Geo::Coder::Free::DB::MaxMind::cities;
21
16
16
16
105909
10
347
use Geo::Coder::Free::Provenance qw(GEOCODER);
22
16
16
16
3202
2228
401
use Geo::Location::Predicate qw(_valid_location);
23
16
16
16
36
12
147
use Geo::Location::Point;
24
16
16
16
2864
31629
216
use Module::Info;
25
16
16
16
30
11
373
use Carp;
26
16
16
16
24
13
111
use File::Spec;
27
16
16
16
23
14
200
use Params::Get;
28
16
16
16
23
11
128
use Locale::CA;
29
16
16
16
19
15
103
use Locale::US;
30
16
16
16
21
12
117
use CHI;
31
16
16
16
2504
307025
943
use Locale::Country;
32
16
16
16
53
39
401
use Scalar::Util;
33# Imported functions are not methods: without this, $obj->croak() and the like
34# would resolve.  _valid_location is kept: t/edge_cases.t
35# checks that this package holds the rule itself.
36
16
16
16
24
29
58
use namespace::clean -except => [qw(_valid_location)];
37
38our %admin1cache;
39our %admin2cache;     # name -> region code (e.g. 'Kent' -> 'P5')
40our %admin2cache_rev; # region code -> name (inverse; avoids O(N) scan in _prepare)
41
42# Lazily-initialised Locale singletons - constructing Locale::US / Locale::CA
43# on every geocode call for US/Canadian addresses was the top object-churn
44# hotspot.  These are module-level my-variables (not exported) so they survive
45# across object instances and avoid the construction cost after the first call.
46my $_locale_us;
47my $_locale_ca;
48
49sub _prepare;
50
51# Some locations aren't found because of inconsistencies in the way things are stored - these are some values I know
52# FIXME: Should be in a configuration file
53my %known_locations = (
54        'Newport Pagnell, Buckinghamshire, England' => {
55                'latitude' => 52.08675,
56                'longitude' => -0.72270
57        },
58);
59
60 - 68
=head1 NAME

Geo::Coder::Free::MaxMind - Provides a geocoding functionality using the MaxMind and GeoNames databases

=head1 VERSION

Version 0.51

=cut
69
70our $VERSION = '0.51';
71
72 - 140
=head1 SYNOPSIS

    use Geo::Coder::Free::MaxMind;

    my $geocoder = Geo::Coder::Free::MaxMind->new();
    my $location = $geocoder->geocode(location => 'Ramsgate, Kent, UK');

=head1 DESCRIPTION

Geo::Coder::Free::MaxMind provides an interface to free databases.

Refer to the source URL for licencing information for these files:
cities.csv is from L<https://www.maxmind.com/en/free-world-cities-database>;
admin1.db is from L<http://download.geonames.org/export/dump/admin1CodesASCII.txt>;
admin2.db is from L<http://download.geonames.org/export/dump/admin2Codes.txt>;

See also L<http://download.geonames.org/export/dump/allCountries.zip>

To significantly speed this up,
gunzip cities.csv and run it through the db2sql script to create an SQLite file.

=head1 METHODS

=head2 new

    $geocoder = Geo::Coder::Free::MaxMind->new();

Takes one optional parameter, directory,
which tells the library where to find the files admin1db, admin2.db and cities.[sql|csv.gz].
If that parameter isn't given, the module will attempt to find the databases, but that can't be guaranteed

There are 3 levels to the Maxmind database.
Here's the method to find the location of Sittingbourne, Kent, England:
1) admin1.db contains admin areas such as counties, states and provinces
   A typical line is:
     US.MD      Maryland        Maryland        4361885
   So a look-up of 'Maryland' will get the concatenated code 'US.MD'
   Note that GB has England, Scotland and Wales at this level, not the counties
     GB.ENG     England England 6269131
   So a look-up of England will give the concatenated code of GB.ENG for use in admin2.db
2) admin2.db contains admin areas drilled down from the admin1 database such as US counties
   Note that GB has counties
   A typical line is:
    GB.ENG.G5   Kent    Kent    3333158
   So a look-up of 'Kent' with a concatenated code to start with 'GB.ENG' will code the region G5 for use in cities.sql
3) cities.sql contains the latitude and longitude of the place we want, so a search for 'sittingbourne' in the
   region 'g5' will give
     gb,sittingbourne,Sittingbourne,G5,41148,51.333333,.75

The admin2.db is far from comprehensive, see Makefile.PL for some entries that are added manually.

Called on an object, it returns a shallow copy with the arguments merged in.
It croaks if the directory doesn't exist.  Other arguments are passed to
L<Database::Abstraction>.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    directory => { type => 'string', optional => 1 }                           # holds databases/; defaults to beside this module
    cache     => { type => 'object', optional => 1, can => ['get', 'set'] }   # CHI-compatible; defaults to a global memory cache

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Coder::Free::MaxMind' }

=cut
141
142sub new
143{
144
116
1
91
        my $class = shift;
145
146        # Handle hash or hashref arguments
147
116
161
        my $args = Params::Get::get_params(undef, @_);
148
149
116
1494
        if(!defined($class)) {
150                # Geo::Coder::Free::Local->new not Geo::Coder::Free::Local::new
151                # carp(__PACKAGE__, ' use ->new() not ::new() to instantiate');
152                # return;
153
154                # FIXME: this only works when no arguments are given
155
0
0
                $class = __PACKAGE__;
156        } elsif(Scalar::Util::blessed($class)) {
157                # If $class is an object, clone it with new arguments
158
0
0
0
0
0
0
                return bless { %{$class}, %{$args} }, ref($class);
159        }
160
161
116
342
        my $directory = $args->{'directory'} || Module::Info->new_from_loaded(__PACKAGE__)->file();
162
116
4473
        $directory =~ s/\.pm$//;
163
164
116
1480
        if(!-d $directory) {
165
0
0
                Carp::croak(ref($class), ": directory $directory doesn't exist");
166        }
167
168        Database::Abstraction::init({
169                cache_duration => '1 day',
170
116
727
                %{$args},
171                directory => File::Spec->catfile($directory, 'databases'),
172
116
102
                cache => $args->{'cache'} || CHI->new(driver => 'Memory', global => 1)
173        });
174
175        # Return the blessed object.  The directory is kept so that misses can be
176        # remembered per data set: two installs, or two directories, do not share
177        # an answer.
178        return bless {
179
116
439639
                cache => $args->{'cache'} || CHI->new(driver => 'Memory', global => 1),
180                directory => $directory,
181        }, $class;
182}
183
184 - 207
=head2 reads

    my $count = $geocoder->reads();

How many times this backend has read its databases since the object was
created - a query on cities, admin1 or admin2.  A lookup answered without
reading anything - from the result cache, or from a hand-curated
C<%known_locations> entry - does not increase it.  C<geocode_many()> takes the
count before and after a value to tell a lookup that cost a read from one that
was answered from a cache.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    # none

=head4 output

    # Output schema (Return::Set)
    { type => 'integer', min => 0 }   # 0 for a new object

=cut
208
209sub reads {
210
47
1
24
        my $self = shift;
211
212
47
88
        return $self->{'reads'} // 0;
213}
214
215sub _read {
216
5
3
        my $self = shift;
217
218
5
2
        $self->{'reads'}++;
219
5
2
        return;
220}
221
222 - 266
=head2 geocode

    $location = $geocoder->geocode(location => $location);

    print 'Latitude: ', $location->lat(), "\n";
    print 'Longitude: ', $location->long(), "\n";

    # TODO:
    # @locations = $geocoder->geocode('Portland, USA');
    # diag 'There are Portlands in ', join (', ', map { $_->{'state'} } @locations);

    # This will return one place in New Brunwsick, not them all
    # TODO: Arguably it should get them all from the database (or at least say the first 100) and return the central location
    my @locations = $geocoder->geocode({ location => 'New Brunswick, Canada' });
    die if(scalar(@locations) != 1);

=head3 provenance

Every result carries C<geocoder>, C<database>, C<confidence> and C<accuracy>
(section 2.1 of F<BI-Phase6-Dependency-Specs.md>).  This backend reports
C<database> as C<MaxMind>.  C<accuracy> is C<city> when the row was chosen by
city name, C<region> when the lookup named only a region or a country - the row
is then the first match in that region, not a city the caller asked for - and
C<locality> for a hand-curated C<%known_locations> entry, whose coordinate is a
town's.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) - also accepts a hashref or a bare location string
    location => { type => 'string' }                 # must contain a comma unless region is given
    region   => { type => 'string', optional => 1 }  # ISO 3166-1 alpha-2 country code hint

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Location::Point', optional => 1 }   # at most one, in any context

Returns nothing when the location has no comma and no region is given, when it
is not found, or when it was already remembered as a miss.  It croaks on a
missing location, on a reference other than a hashref, and on a location that
fails L<Geo::Location::Predicate>'s rule, such as one made only of digits.

=cut
267
268sub geocode {
269
6
1
3432
        my $self = shift;
270
6
5
        my %params;
271
272        # Try hard to support whatever API that the user wants to use
273
6
27
        if(!ref($self)) {
274
0
0
                if(scalar(@_)) {
275
0
0
                        return(__PACKAGE__->new()->geocode(@_));
276                } elsif(!defined($self)) {
277                        # Geo::Coder::Free->geocode()
278
0
0
                        Carp::croak('Usage: ', __PACKAGE__, '::geocode(location => $location|scantext => $text)');
279                } elsif($self eq __PACKAGE__) {
280
0
0
                        Carp::croak("Usage: $self", '::geocode(location => $location|scantext => $text)');
281                }
282
0
0
                return(__PACKAGE__->new()->geocode($self));
283        } elsif(ref($self) eq 'HASH') {
284
0
0
                return(__PACKAGE__->new()->geocode($self));
285        } elsif(ref($_[0]) eq 'HASH') {
286
1
1
1
1
                %params = %{$_[0]};
287        # } elsif(ref($_[0]) && (ref($_[0] !~ /::/))) {
288        } elsif(ref($_[0])) {
289
0
0
                Carp::croak('Usage: ', __PACKAGE__, '::geocode(location => $location|scantext => $text)');
290        } elsif(scalar(@_) && (scalar(@_) % 2 == 0)) {
291
5
5
                %params = @_;
292        } else {
293
0
0
                $params{'location'} = shift;
294        }
295
296        my $location = $params{location}
297
6
10
                or Carp::croak('Usage: geocode(location => $location)');
298
299        # Fail when the input is just a set of numbers.  The rule is the one
300        # Geo::Location::Predicate defines and the facade applies too, so what this
301        # backend refuses cannot drift away from what valid_location() answers.
302
6
9
        if(!_valid_location($location)) {
303
2
22
                Carp::croak('Usage: ', __PACKAGE__, ": invalid input to geocode(), $location");
304        }
305
306
4
18
        if($location =~ /^(.+),\s*Washington\s*DC,(.+)$/) {
307
0
0
                $location = "$1, Washington, DC, $2";
308        }
309
310
4
8
        if(my $rc = $known_locations{$location}) {
311                # return $known_locations{$location};
312                return Geo::Location::Point->new({
313                        'lat' => $rc->{'latitude'},
314                        'long' => $rc->{'longitude'},
315                        'lon' => $rc->{'longitude'},
316
1
6
                        'lng' => $rc->{'longitude'},
317                        'location' => $location,
318                        'geocoder' => GEOCODER,
319                        'database' => 'MaxMind',
320                        # A hand-curated entry: the string matched it exactly, and the
321                        # coordinate it carries is a town's, not an address's.
322                        'confidence' => 1.0,
323                        'accuracy' => 'locality'
324                });
325        }
326
327
3
11
        return unless(($location =~ /,/) || $params{'region'}); # Not well formed, or an attempt to find the location of an entire country
328
329        # Check cache first
330
2
12
        my $cached_result = $self->{'cache'}->get($location);
331
1
99
        return $cached_result if($cached_result);
332
333        # A miss is remembered for the whole process.  The key is the location as
334        # given (it is rewritten below into its component parts) together with the
335        # region hint, which is part of what the lookup asks: a bare town name in
336        # one region can be found where the same name in another is not.
337
1
6
        my $miss_scope = Geo::Coder::Free::MissCache->scope_for('MaxMind', $self->{'directory'} // '');
338
1
4
        my $miss_key = join("\0", $location, $params{'region'} // '');
339
1
3
        return if Geo::Coder::Free::MissCache->missed($miss_scope, $miss_key);
340
341
1
3
        my $county;
342        my $state;
343
1
0
        my $country;
344
1
0
        my $country_code;
345
1
0
        my $concatenated_codes;
346
1
0
        my $region_only;
347
348
1
5
        if($location =~ /^([\w\s\-]+),([\w\s]+),([\w\s]+)?$/) {
349                # Turn 'Ramsgate, Kent, UK' into 'Ramsgate'
350
1
1
                $location = $1;
351
1
6
                $county = $2;
352
1
1
                $country = $3;
353
1
1
                $location =~ s/\-/ /g;
354
1
2
                $county =~ s/^\s//g;
355
1
1
                $county =~ s/\s$//g;
356
1
2
                $country =~ s/^\s//g;
357
1
2
                $country =~ s/\s$//g;
358
1
3
                if($location =~ /^St\.? (.+)/) {
359
0
0
                        $location = "Saint $1";
360                }
361
1
3
                if(($country =~ /^(Canada|United States|USA|US)$/)) {
362
0
0
                        $state = $county;
363
0
0
                        $county = undef;
364                }
365        } elsif($location =~ /^([\w\s\-]+),([\w\s]+),([\w\s]+),\s*(Canada|United States|USA|US)?$/) {
366
0
0
                $location = $1;
367
0
0
                $county = $2;
368
0
0
                $state = $3;
369
0
0
                $country = $4;
370
0
0
                $county =~ s/^\s//g;
371
0
0
                $county =~ s/\s$//g;
372
0
0
                $state =~ s/^\s//g;
373
0
0
                $state =~ s/\s$//g;
374                # $country =~ s/^\s//g;
375
0
0
                $country =~ s/\s$//g;
376        } elsif($location =~ /^[\w\s-],[\w\s-]/) {
377
0
0
                Carp::carp(__PACKAGE__, ": can't parse and handle $location");
378
0
0
                Geo::Coder::Free::MissCache->remember($miss_scope, $miss_key);
379
0
0
                return;
380        } elsif(($location =~ /^[\w\s-]+$/) && (my $region = $params{'region'})) {
381
0
0
                $location =~ s/^\s//g;
382
0
0
                $location =~ s/\s$//g;
383
0
0
                $country = uc($region);
384        } elsif($location =~ /^([\w\s-]+),\s*(\w+)$/) {
385                # e.g. a county in the UK or a state in the US
386
0
0
                $county = $1;
387
0
0
                $country = $2;
388
0
0
                $county =~ s/^\s//g;
389
0
0
                $county =~ s/\s$//g;
390                # $country =~ s/^\s//g;
391
0
0
                $country =~ s/\s$//g;
392
0
0
                $region_only = 1;       # Will only return one match, not every match in the region
393        } else {
394                # Carp::croak(__PACKAGE__, ' only supports towns, not full addresses');
395
0
0
                Geo::Coder::Free::MissCache->remember($miss_scope, $miss_key);
396
0
0
                return;
397        }
398
1
0
        my $countrycode;
399
1
2
        if($country) {
400
1
4
                if(defined($country) && (($country eq 'UK') || ($country eq 'United Kingdom') || ($country eq 'England'))) {
401
1
1
                        $country = 'Great Britain';
402
1
0
                        $concatenated_codes = 'GB';
403                }
404
1
3
                $countrycode = country2code($country);
405                # if($county && $countrycode) {
406                # }
407
408
1
37
                if($state && $admin1cache{$state}) {
409
0
0
                        $concatenated_codes = $admin1cache{$state};
410                } elsif($admin1cache{$country} && !defined($state)) {
411
0
0
                        $concatenated_codes = $admin1cache{$country};
412                } else {
413
1
4
                        $self->{'admin1'} //= Geo::Coder::Free::DB::MaxMind::admin1->new(no_entry => 1) or Carp::croak("Can't open the admin1 database");
414
415
1
2
                        $self->_read();
416
1
1
                        if(my $admin1 = $self->{'admin1'}->fetchrow_hashref(asciiname => $country)) {
417
0
0
                                $concatenated_codes = $admin1->{'concatenated_codes'};
418
0
0
                                $admin1cache{$country} = $concatenated_codes;
419                        } elsif($state) {
420
0
0
                                $concatenated_codes = uc($countrycode);
421
0
0
                                if($state =~ /^[A-Z]{2}$/) {
422
0
0
                                        $concatenated_codes .= ".$state";
423                                } else {
424
0
0
                                        $country_code = $concatenated_codes;
425
0
0
                                        $self->_read();
426
0
0
0
0
                                        my @admin1s = @{$self->{'admin1'}->selectall_hashref(asciiname => $state)};
427
0
0
                                        foreach my $admin1(@admin1s) {
428
0
0
                                                if($admin1->{'concatenated_codes'} =~ /^\Q$concatenated_codes\E\./i) {
429
0
0
                                                        $concatenated_codes = $admin1->{'concatenated_codes'};
430
0
0
                                                        last;
431                                                }
432                                        }
433                                }
434
0
0
                                $admin1cache{$state} = $concatenated_codes;
435                        } elsif($countrycode) {
436
1
6
                                $concatenated_codes = uc($countrycode);
437
1
1
                                $admin1cache{$country} = $concatenated_codes;
438                        } elsif(Locale::Country::code2country($country)) {
439
0
0
                                $concatenated_codes = uc($country);
440
0
0
                                $admin1cache{$country} = $concatenated_codes;
441                        }
442                }
443        }
444
1
2
        if(!defined($concatenated_codes)) {
445
0
0
                Geo::Coder::Free::MissCache->remember($miss_scope, $miss_key);
446
0
0
                return;
447        }
448
449
1
1
        my @admin2s;
450        my $region;
451
1
0
        my @regions;
452
1
6
        if($country =~ /^(United States|USA|US)$/) {
453
0
0
                if($county && (length($county) > 2)) {
454
0
0
                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($county)}) {
455
0
0
                                $county = $twoletterstate;
456                        }
457                }
458
0
0
                if($state && (length($state) > 2)) {
459
0
0
                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
460
0
0
                                $state = $twoletterstate;
461                        }
462                }
463        } elsif(($country eq 'Canada') && $state && (length($state) > 2)) {
464
0
0
                if(($_locale_ca //= Locale::CA->new())->{province2code}{uc($state)}) {
465                        # FIXME:  I can't see that province locations are stored in cities.csv
466
0
0
                        if(!defined($location)) {       # OK if searching for a city, that works
467
0
0
                                Geo::Coder::Free::MissCache->remember($miss_scope, $miss_key);
468
0
0
                                return;
469                        }
470                }
471        }
472
473
1
2
        $self->{'admin2'} //= Geo::Coder::Free::DB::MaxMind::admin2->new(no_entry => 1) or Carp::croak("Can't open the admin2 database");
474
475
1
9
        if(defined($county) && ($county =~ /^[A-Z]{2}$/) && ($country =~ /^(United States|USA|US)$/)) {
476                # US state. Not Canadian province.
477
0
0
                $region = $county;
478        } elsif($county && $admin1cache{$county}) {
479
0
0
                $region = $admin1cache{$county};
480        } elsif($county && $admin2cache{$county}) {
481
0
0
                $region = $admin2cache{$county};
482        } elsif(defined($state) && $admin2cache{$state} && !defined($county)) {
483
0
0
                $region = $admin2cache{$state};
484        } else {
485
1
5
                if(defined($county) && ($county eq 'London')) {
486
0
0
                        $self->_read();
487
0
0
                        @admin2s = $self->{'admin2'}->selectall_hash(asciiname => $location);
488                } elsif(defined($county)) {
489
1
2
                        $self->_read();
490
1
2
                        @admin2s = $self->{'admin2'}->selectall_hash(asciiname => $county);
491                }
492
1
2
                foreach my $admin2(@admin2s) {
493
0
0
                        if($admin2->{'concatenated_codes'} =~ $concatenated_codes) {
494
0
0
                                $region = $admin2->{'concatenated_codes'};
495
0
0
                                if($region =~ /^[A-Z]{2}\.([A-Z]{2})\./) {
496
0
0
                                        my $rc = $1;
497
0
0
                                        if(defined($state) && ($state =~ /^[A-Z]{2}$/)) {
498
0
0
                                                if($state eq $rc) {
499
0
0
                                                        $region = $rc;
500
0
0
                                                        @regions = ();
501
0
0
                                                        last;
502                                                }
503                                        } else {
504
0
0
                                                push @regions, $region;
505
0
0
                                                push @regions, $rc;
506                                        }
507                                } else {
508
0
0
                                        push @regions, $region;
509                                }
510                        }
511                }
512
1
2
                if($state && !defined($region)) {
513
0
0
                        if($state =~ /^[A-Z]{2}$/) {
514
0
0
                                $region = $state;
515
0
0
                                @regions = ();
516                        } else {
517
0
0
                                $self->_read();
518
0
0
                                @admin2s = $self->{'admin2'}->selectall_hash(asciiname => $state);
519
0
0
                                foreach my $admin2(@admin2s) {
520
0
0
                                        if($admin2->{'concatenated_codes'} =~ $concatenated_codes) {
521
0
0
                                                $region = $admin2->{'concatenated_codes'};
522
0
0
                                                last;
523                                        }
524                                }
525                        }
526                }
527        }
528
529
1
2
        if((scalar(@regions) == 0) && !defined($region)) {
530                # e.g. Unitary authorities in the UK
531                # admin[12].db columns are labelled ['concatenated_codes', 'name', 'asciiname', 'geonameId']
532
1
1
                $self->_read();
533
1
2
                @admin2s = $self->{'admin2'}->selectall_hash(asciiname => $location);
534
1
5
                if((scalar(@admin2s) == 0) && ($country =~ /^(Canada|United States|USA|US)$/) && ($location !~ /\sCounty/i)) {
535
0
0
                        $location .= ' County';
536
0
0
                        $self->_read();
537
0
0
                        @admin2s = $self->{'admin2'}->selectall_hash(asciiname => $location);
538                }
539
1
3
                if(scalar(@admin2s) && defined($admin2s[0]->{'concatenated_codes'})) {
540
0
0
                        foreach my $admin2(@admin2s) {
541
0
0
                                my $concat = $admin2->{'concatenated_codes'};
542
0
0
                                if($concat =~ /^CA\.(\d\d)\./) {
543                                        # Canadian provinces are not stored in the same way as US states
544
0
0
                                        $region = $1;
545
0
0
                                        last;
546                                } elsif($concat =~ $concatenated_codes) {
547
0
0
                                        $region = $concat;
548
0
0
                                        last;
549                                }
550                        }
551                } elsif(defined($county)) {
552                        # e.g. states in the US
553
1
1
                        if(!defined($self->{'admin1'})) {
554
0
0
                                $self->{'admin1'} = Geo::Coder::Free::DB::MaxMind::admin1->new(no_entry => 1) or Carp::croak("Can't open the admin1 database");
555                        }
556
1
4
                        $self->_read();
557
1
3
                        my @admin1s = $self->{'admin1'}->selectall_hash(asciiname => $county);
558
1
2
                        foreach my $admin1(@admin1s) {
559
0
0
                                if($admin1->{'concatenated_codes'} =~ /^\Q$concatenated_codes\E\./i) {
560
0
0
                                        $region = $admin1->{'concatenated_codes'};
561
0
0
                                        if(scalar(@admin1s) == 1) {
562
0
0
                                                $admin1cache{$county} = $region;
563                                        }
564
0
0
                                        last;
565                                }
566                        }
567                }
568        }
569
570
1
2
        if(!defined($self->{'cities'})) {
571                $self->{'cities'} = Geo::Coder::Free::DB::MaxMind::cities->new(
572
0
0
                        cache => $self->{cache} || CHI->new(driver => 'Memory', datastore => {}),
573                        no_entry => 1,
574                );
575        }
576
577
1
1
        my $options;
578
1
3
        if(defined($county) && ($county =~ /^[A-Z]{2}$/) && ($country =~ /^(United States|USA|US)$/)) {
579
0
0
                $options = { Country => 'us' };
580        } else {
581
1
1
                if($region_only) {
582
0
0
                        $options = {};
583                } else {
584
1
2
                        $options = { City => lc($location) };
585
1
2
                        $options->{'City'} =~ s/,\s*\w+$//;
586                }
587        }
588
1
1
        if($region) {
589
0
0
                $region =~ s/^.*\.//;    # keep only the rightmost dot-delimited component
590
0
0
                $options->{'Region'} = $region;
591
0
0
                if($country_code) {
592
0
0
                        $options->{'Country'} = lc($country_code);
593                }
594                # If there's more than one match, don't cache as we don't
595                # know which one will be matched later
596
0
0
                if(scalar(@admin2s) == 1) {
597
0
0
                        if($state) {
598
0
0
                                $admin2cache{$state}     = $region;
599
0
0
                                $admin2cache_rev{$region} = $state;
600                        } elsif($county) {
601
0
0
                                $admin2cache{$county}    = $region;
602
0
0
                                $admin2cache_rev{$region} = $county;
603                        }
604                }
605        }
606
607
1
1
        my $confidence = 0.5;
608
1
1
        if(my $c = $params{'region'}) {
609
0
0
                $options->{'Country'} = lc($c);
610
0
0
                $confidence = 0.1;
611        } elsif($countrycode) {
612
1
1
                $options->{'Country'} = $countrycode;
613
1
1
                $confidence = 0.1;
614        }
615        # This case nonsense is because DBD::CSV changes the columns to lowercase, whereas DBD::SQLite does not
616        # if(wantarray && (!$options->{'City'}) && !$region_only) {
617        # if(0) {       # We don't need to find all the cities in a state, which is what this would do
618                # my @rc = $self->{'cities'}->selectall_hash($options);
619                # if(scalar(@rc) == 0) {
620                        # if((!defined($region)) && !defined($params{'region'})) {
621                                # # Add code for this area to Makefile.PL and rebuild
622                                # Carp::carp(__PACKAGE__, ": didn't determine region from $location");
623                                # return;
624                        # }
625                        # # This would return all of the cities in the wrong region
626                        # if($countrycode) {
627                                # @rc = $self->{'cities'}->selectall_hash('Region' => ($region || $params{'region'}), 'Country' => $countrycode);
628                                # if(scalar(@rc) == 0) {
629                                        # return;
630                                # }
631                        # }
632                # }
633                # foreach my $city(@rc) {
634                        # if($city->{'Latitude'}) {
635                                # $city->{'latitude'} = delete $city->{'Latitude'};
636                                # $city->{'longitude'} = delete $city->{'Longitude'};
637                        # }
638                        # if($city->{'Country'}) {
639                                # $city->{'country'} = uc(delete $city->{'Country'});
640                        # }
641                        # if($city->{'Region'}) {
642                                # $city->{'state'} = uc(delete $city->{'Region'});
643                        # }
644                        # if($city->{'City'}) {
645                                # $city->{'city'} = uc(delete $city->{'AccentCity'});
646                                # delete $city->{'City'};
647                                # # Less likely to get false positives with long words
648                                # if(length($city->{'city'}) > 10) {
649                                        # if($confidence <= 0.8) {
650                                                # $confidence += 0.2;
651                                        # } else {
652                                                # $confidence = 1.0;
653                                        # }
654                                # }
655                        # }
656                        # $city->{'confidence'} = $confidence;
657                        # my $l = $options->{'City'};
658                        # if($options->{'Region'}) {
659                                # $l .= ', ' . $options->{'Region'};
660                        # }
661                        # if($options->{'Country'}) {
662                                # $l .= ', ' . ucfirst($options->{'Country'});
663                        # }
664                        # $city->{'location'} = $l;
665                # }
666                # # return @rc;
667                # my @locations;
668                #
669                # foreach my $l(@rc) {
670                        # if(exists($l->{'latitude'})) {
671                                # push @locations, Geo::Location::Point->new({
672                                        # 'lat' => $l->{'latitude'},
673                                        # 'long' => $l->{'longitude'},
674                                        # 'lon' => $l->{'longitude'},
675                                        # 'location' => $location,
676                                        # 'database' => 'MaxMind',
677                                        # 'maxmind' => $l,
678                                # });
679                        # # } else {
680                                # # Carp::carp(__PACKAGE__, ": $location has latitude of 0");
681                                # # return;
682                        # }
683                # }
684                #
685                # return @locations;
686        # }
687
1
1
        $self->_read();
688
1
2
        my $city = $self->{'cities'}->fetchrow_hashref($options);
689
1
5
        if(!defined($city)) {
690
0
0
                foreach $region(@regions) {
691
0
0
                        $region =~ s/^.*\.//;    # keep only the rightmost dot-delimited component
692
0
0
                        if($country =~ /^(United States|USA|US)$/) {
693
0
0
                                next unless($region =~ /^[A-Z]{2}$/);   # In the US, the regions are the states
694                        }
695
0
0
                        $options->{'Region'} = $region;
696
0
0
                        $self->_read();
697
0
0
                        $city = $self->{'cities'}->fetchrow_hashref($options);
698
0
0
                        last if(defined($city));
699                }
700        }
701
702
1
4
        if(defined($city) && defined($city->{'Latitude'})) {
703                # Cache and return result
704
1
5
                delete $city->{'Region'} if(defined($city->{'Region'}) && ($city->{'Region'} =~ /^[A-Z]\d$/)); # E.g. Region = G5
705
1
2
                delete $city->{'Population'} if(defined($city->{'Population'}) && (length($city->{'Population'}) == 0));
706                my $rc = Geo::Location::Point->new({
707
1
4
                        %{$city},
708                        ('geocoder' => GEOCODER,
709                         'database' => 'MaxMind',
710                         'confidence' => $confidence,
711                         # A row chosen for its city name is a city; one chosen by region or
712                         # country alone is the first match in that region.
713
1
1
                         'accuracy' => $options->{'City'} ? 'city' : 'region')
714                });
715
1
25
                $self->{cache}->set($location, $rc);
716
1
233
                return $rc;
717        }
718        # return $city;
719
0
        Geo::Coder::Free::MissCache->remember($miss_scope, $miss_key);
720
0
        return;
721}
722
723 - 749
=head2  reverse_geocode

    $location = $geocoder->reverse_geocode(latlng => '37.778907,-122.39732');

Returns a string, or undef if it can't be found.
The nearest place within 0.01 degrees is chosen; in list context every place
within that distance is returned.  Separate C<lat> and C<lon> (or C<long>) are
accepted instead of C<latlng>.  It croaks if a coordinate is missing or is not
a decimal number.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) - also accepts a hashref or a bare "lat,long" string
    latlng => { type => 'string', optional => 1 }   # "lat,long"
    lat    => { type => 'number', optional => 1 }   # with lon or long, when latlng isn't given
    lon    => { type => 'number', optional => 1 }
    long   => { type => 'number', optional => 1 }

=head4 output

    # Output schema (Return::Set)
    # scalar context: { type => 'string', optional => 1 }
    # list context:   { type => 'arrayref', of => 'string' }

=cut
750
751sub reverse_geocode
752{
753
0
1
        my $self = shift;
754
0
        my %params;
755
756        # Try hard to support whatever API that the user wants to use
757
0
        if(!ref($self)) {
758
0
                if(scalar(@_)) {
759
0
                        return(__PACKAGE__->new()->reverse_geocode(@_));
760                } elsif(!defined($self)) {
761                        # Geo::Coder::Free->reverse_geocode()
762
0
                        Carp::croak('Usage: ', __PACKAGE__, '::reverse_geocode(latlng => "$lat,$long")');
763                } elsif($self eq __PACKAGE__) {
764
0
                        Carp::croak("Usage: $self", '::reverse_geocode(latlng => "$lat,$long")');
765                }
766
0
                return(__PACKAGE__->new()->reverse_geocode($self));
767        } elsif(ref($self) eq 'HASH') {
768
0
                return(__PACKAGE__->new()->reverse_geocode($self));
769        } elsif(ref($_[0]) eq 'HASH') {
770
0
0
                %params = %{$_[0]};
771        # } elsif(ref($_[0]) && (ref($_[0] !~ /::/))) {
772        } elsif(ref($_[0])) {
773
0
                Carp::croak('Usage: ', __PACKAGE__, '::reverse_geocode(latlng => "$lat,$long")');
774        } elsif(scalar(@_) && (scalar(@_) % 2 == 0)) {
775
0
                %params = @_;
776        } else {
777
0
                $params{'latlng'} = shift;
778        }
779
780
0
        my $latlng = $params{'latlng'};
781
782
0
        my $latitude;
783        my $longitude;
784
785
0
        if($latlng) {
786
0
                ($latitude, $longitude) = split(/,/, $latlng);
787        } else {
788
0
                $latitude //= $params{'lat'};
789
0
                $longitude //= $params{'lon'};
790
0
                $longitude //= $params{'long'};
791        }
792
793
0
        if((!defined($latitude)) || !defined($longitude)) {
794
0
                Carp::croak('Usage: ', __PACKAGE__, '::reverse_geocode(latlng => "$lat,$long")');
795        }
796
797
0
        if(!defined($self->{'cities'})) {
798                $self->{'cities'} = Geo::Coder::Free::DB::MaxMind::cities->new(
799
0
                        cache => $self->{cache} || CHI->new(driver => 'Memory', datastore => {}),
800                        no_entry => 1,
801                );
802        }
803
804        # Validate coordinates before interpolating into SQL - prevents injection
805        # if a caller passes a non-numeric latlng string.
806
0
        for my $c ($latitude, $longitude) {
807
0
                Carp::croak('Invalid coordinate value for reverse_geocode')
808                        unless defined($c) && $c =~ /^-?\d+\.?\d*$/;
809        }
810
811
0
        if(wantarray) {
812
0
                $self->_read();
813
0
                my @locs = $self->{'cities'}->execute("SELECT * FROM cities WHERE ((ABS(Latitude - $latitude)) < 0.01) AND ((ABS(Longitude - $longitude)) < 0.01)");
814
0
                foreach my $loc(@locs) {
815
0
                        $self->_prepare($loc);
816                }
817
0
0
                return map { Geo::Location::Point->new($_)->as_string() } @locs;
818        }
819
820        # Performance: the former code issued 5 sequential SQL queries with ever-
821        # widening radii (0.000001 -> 0.01).  Replace with a single query that
822        # orders by Manhattan distance and caps at the maximum radius (0.01 degrees).
823        # This eliminates 4 round-trips to SQLite on the common miss-until-last case.
824
0
        $self->_read();
825
0
        my $rc = $self->{'cities'}->execute(
826                "SELECT * FROM cities" .
827                " WHERE ABS(Latitude - $latitude) < 0.01 AND ABS(Longitude - $longitude) < 0.01" .
828                " ORDER BY (ABS(Latitude - $latitude) + ABS(Longitude - $longitude)) ASC" .
829                " LIMIT 1"
830        );
831
0
        if($rc) {
832
0
                $self->_prepare($rc);
833
0
                return Geo::Location::Point->new($rc)->as_string();
834        }
835
0
        return;
836}
837
838# Change 'Charing Cross, P5, Gb' to 'Charing Cross, London, Gb'
839sub _prepare {
840
0
        my ($self, $loc) = @_;
841
842
0
        if(my $region = $loc->{'Region'}) {
843                # O(1) reverse lookup - the former O(N) `while each %admin2cache` scan
844                # walked the entire cache comparing values.  %admin2cache_rev is kept in
845                # sync at every write site, giving a constant-time code->name translation.
846
0
                my $county = $admin2cache_rev{$region};
847
0
                if($county) {
848
0
                        $loc->{'Region'} = $county;
849                } else {
850
0
                        $self->{'admin2'} //= Geo::Coder::Free::DB::MaxMind::admin2->new(no_entry => 1) or Carp::croak("Can't open the admin2 database");
851
852                        # Validate both tokens before interpolating into SQL.
853                        # $loc->{'Country'} is DB-sourced (ISO 3166-1 alpha-2 = two
854                        # uppercase letters) and $region is user-supplied.  Unvalidated
855                        # interpolation into a LIKE predicate would allow SQL injection.
856
0
                        my $_country = uc($loc->{'Country'} // '');
857
0
                        my $_reg     = uc($region // '');
858
0
                        my $row;
859
0
                        if($_country =~ /^[A-Z]{2}$/ && $_reg =~ /^[A-Z0-9]{1,10}$/) {
860
0
                                $self->_read();
861
0
                                $row = $self->{'admin2'}->execute("SELECT name FROM admin2 WHERE concatenated_codes LIKE '$_country.%.$_reg' LIMIT 1");
862                        }
863
0
                        if(ref($row) && $row->{'name'}) {
864                                # Write to both forward and reverse caches in one go
865
0
                                $admin2cache{$row->{'name'}}  = $region;
866
0
                                $admin2cache_rev{$region}      = $row->{'name'};
867
0
                                $loc->{'Region'}               = $row->{'name'};
868                        }
869                }
870        }
871}
872
873 - 889
=head2  ua

Does nothing, here for compatibility with other geocoders

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    # any arguments are accepted and ignored

=head4 output

    # Output schema (Return::Set)
    # nothing: an empty list, or undef in scalar context

=cut
890
891
1
sub ua {
892}
893
894 - 941
=head1 AUTHOR

Nigel Horne, C<< <njh@bandsman.co.uk> >>

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.

=head1 BUGS

Lots of lookups fail at the moment.

The MaxMind data only contains cities.

Can't parse and handle "London, England".

The database contains Canadian cities, but not provinces, so a search for "New Brunswick, Canada" won't work

The GeoNames admin databases are in this class, they should be in Geo::Coder::GeoNames.

The data at
L<https://github.com/apache/commons-csv/blob/master/src/test/resources/org/apache/commons/csv/perf/worldcitiespop.txt.gz?raw=true>
are 7 years out of date,
and are inconsistent with the Geonames database.

If you search for something like "Sheppy, Kent, England" in list context,
it returns them all.
That's a lot!
It should be limited to,
say 10 results (that number should be tuneable, and be a LIMIT in DB.pm),
and as the correct spelling in Sheppey, arguably it should return nothing.

=head1 SEE ALSO

VWF, MaxMind and geonames.

=head1 LICENSE AND COPYRIGHT

Copyright 2017-2025 Nigel Horne.

The program code is released under the following licence: GPL for personal use on a single computer.
All other users (including Commercial, Charity, Educational, Government)
must apply in writing for a licence for use from Nigel Horne at `<njh at nigelhorne.com>`.

This product includes GeoLite2 data created by MaxMind, available from
L<https://www.maxmind.com/en/home>.
(Note that this currently gives a 403 error - I need to find the latest URL).

=cut
942
9431;