File Coverage

File:blib/lib/Geo/Coder/Free/OpenAddresses.pm
Coverage:24.0%

linestmtbrancondsubpodtimecode
1package Geo::Coder::Free::OpenAddresses;
2
3# Includes both openaddresses and Whos On First data
4
5
17
17
17
71711
16
221
use strict;
6
17
17
17
26
15
326
use warnings;
7
17
17
17
30
5874
36
use autodie qw(:all);
8
9
17
17
17
40237
14
347
use Geo::Coder::Free::Provenance qw(GEOCODER);
10
17
17
17
36
11
317
use Geo::Coder::Free::Utils qw(_abbreviate _normalize);
11
17
17
17
31
315
215
use Geo::Location::Predicate qw(_valid_location);
12
17
17
17
25
14
138
use Geo::Coder::Free::MissCache;
13
17
17
17
1972
21
58
use Geo::Coder::Free::DB::OpenAddr;     # SQLite database
14
17
17
17
136620
22
54
use Geo::Coder::Free::DB::openaddresses;        # The original CSV files
15
17
17
17
114077
6522
240
use Geo::Hash;
16
17
17
17
37
11
139
use Geo::Location::Point;
17
17
17
17
46
2325
121
use Module::Info;
18
17
17
17
23
11
357
use Carp;
19
17
17
17
27
11
129
use File::Spec;
20
17
17
17
2521
3861
364
use File::pfopen;
21
17
17
17
37
3300
152
use Locale::CA;
22
17
17
17
27
357
126
use Locale::US;
23
17
17
17
31
39206
415
use Locale::SubCountry;
24
17
17
17
25
10
139
use CHI;
25
17
17
17
23
19809
137
use Lingua::EN::AddressParse;
26
17
17
17
25
19868
680
use Locale::Country;
27
17
17
17
31
24185
218
use Geo::StreetAddress::US;
28
17
17
17
23
11
215
use Digest::MD5;
29
17
17
17
26
12
514
use Encode;
30
17
17
17
25
12
182
use Params::Get;
31
17
17
17
28
7
387
use Storable;
32# Imported functions are not methods: without this, $obj->croak() and the like
33# would resolve.  _valid_location is kept: t/edge_cases.t
34# checks that this package holds the rule itself.
35
17
17
17
25
11
48
use namespace::clean -except => [qw(_valid_location)];
36
37# Some locations aren't found because of inconsistencies in the way things are stored - these are some values I know
38# FIXME: Should be in a configuration file
39our %known_locations = (
40        'Newport Pagnell, Buckinghamshire, England' => {
41                'latitude' => 52.08675,
42                'longitude' => -0.72270
43        },
44);
45
46# Lazily-initialised Locale singletons.  Locale::US->new() and
47# Locale::CA->new() were being called up to five times per geocode()
48# invocation for US/Canadian addresses.  These module-level variables
49# persist across all object instances and eliminate repeated construction.
50my $_locale_us;
51my $_locale_ca;
52
53
17
17
17
9022
12
379
use constant    LIBPOSTAL_UNKNOWN => 0;
54
17
17
17
27
9
292
use constant    LIBPOSTAL_INSTALLED => 1;
55
17
17
17
23
12
71795
use constant    LIBPOSTAL_NOT_INSTALLED => -1;
56our $libpostal_is_installed = LIBPOSTAL_UNKNOWN;
57
58 - 67
=head1 NAME

Geo::Coder::Free::OpenAddresses -
Provides a geocoding functionality to a local SQLite database containing geo-coding data.

=head1 VERSION

Version 0.51

=cut
68
69our $VERSION = '0.51';
70
71 - 149
=head1 SYNOPSIS

    use Geo::Coder::Free::OpenAddresses;

    # Use a local download of http://results.openaddresses.io/
    my $geocoder;
    if(my $openaddr = $ENV{'OPENADDR_HOME'}) {
        $geocoder = Geo::Coder::Free::OpenAddresses->new(openaddr => $openaddr);
    } else {
        $geocoder = Geo::Coder::Free::OpenAddresses->new(openaddr => '/usr/share/geo-coder-free/data');
    }
    my $location = $geocoder->geocode(location => '1600 Pennsylvania Avenue NW, Washington DC, USA');

    my @matches = $geocoder->geocode({ scantext => 'arbitrary text', region => 'GB' });

=head1 DESCRIPTION

Geo::Coder::Free::OpenAddresses provides an interface to the free geolocation databases at
L<http://results.openaddresses.io>,
L<https://github.com/whosonfirst-data>,
L<https://github.com/dr5hn/countries-states-cities-database.git> and
L<https://download.geofabrik.de/europe-latest.osm.bz2>.
The SQLite database is in a file held in $OPENADDR_HOME/openaddresses.sql.

Refer to the source URL for licencing information for these files.

To install,
run the createdatabases.PL script which imports the data into an SQLite database.
This process will take some time.

=head2 Optional modules

Two modules are used to parse an address when they are installed, and the
lookup goes on without them when they are not:

=over 4

=item * L<Geo::Address::Parser> splits a US address into its fields before the
database is searched for them.

=item * L<Geo::libpostal> is tried last, for US and Canadian addresses the
other steps could not parse.  The first call to it in a process loads
libpostal's model, which is several gigabytes, so that call can take several
seconds and the process then holds that memory.  Pass C<< libpostal => 0 >> to
C<new()> to leave it unused, for example in a short-lived or memory-constrained
process, or in a test with a time limit.

=back

=head1 METHODS

=head2 new

    $geocoder = Geo::Coder::Free::OpenAddresses->new(openaddr => $ENV{'OPENADDR_HOME'});

Takes an optional parameter "openaddr", which is the directory of the file
openaddresses.sql.

Takes an optional parameter cache, which points to an object that understands get() and set() messages to store data in

Takes an optional parameter libpostal.  When it is false, L<Geo::libpostal> is
never loaded or called, even if it is installed; see L</Optional modules>.
It defaults to true.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    openaddr  => { type => 'scalar' }                                          # path to the directory containing openaddresses.sql
    cache     => { type => 'object', optional => 1, can => ['get', 'set'] }   # CHI-compatible cache object
    libpostal => { type => 'boolean', optional => 1, default => 1 }           # false: never use Geo::libpostal

=head4 output

    # Output schema (Return::Set)
    { type => 'object', isa => 'Geo::Coder::Free::OpenAddresses' }

=cut
150
151sub new {
152
37
1
149052
        my($proto, %param) = @_;
153
37
75
        my $class = ref($proto) || $proto;
154
155        # Geo::Coder::Free->new not Geo::Coder::Free::new
156
37
43
        return unless($class);
157
158
37
42
        if(my $openaddr = $param{'openaddr'}) {
159
35
327
                Carp::croak(__PACKAGE__, ": Can't find the directory $openaddr")
160                        if((!-d $openaddr) || (!-r $openaddr));
161                return bless {
162                        openaddr => $openaddr,
163                        cache => $param{'cache'},
164
30
141
                        libpostal => $param{'libpostal'} // 1,
165                }, $class;
166        }
167
2
9
        Carp::croak(__PACKAGE__, ": Usage: new(openaddr => '/path/to/openaddresses')");
168}
169
170 - 191
=head2 reads

    my $count = $geocoder->reads();

How many times this backend has read its database since the object was created.
A lookup answered from the result cache or from the shared miss cache does not
increase it, so C<geocode_many()> can take the count before and after a value to
tell a lookup that cost a read from one that was answered from a cache.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    # none

=head4 output

    # Output schema (Return::Set)
    { type => 'integer', min => 0 }   # 0 for a new object

=cut
192
193sub reads {
194
7
1
508
        my $self = shift;
195
196
7
14
        return $self->{'reads'} // 0;
197}
198
199 - 248
=head2 geocode

    $location = $geocoder->geocode(location => $location);

    print 'Latitude: ', $location->lat(), "\n";
    print 'Longitude: ', $location->long(), "\n";

    # TODO:
    # @locations = $geocoder->geocode('Portland, USA');
    # diag 'There are Portlands in ', join (', ', map { $_->{'state'} } @locations);

    @locations = $geo_coder->geocode(scantext => 'arbitrary text', region => 'US', ignore_words => [ 'foo', 'bar' ]);

When looking for a house number in a street, if that address isn't found but that
street is found, a place in the street is given.
So "106 Wells Street, Fort Wayne, Allen, Indiana, USA" isn't found, a match for
"Wells Street, Fort Wayne, Allen, Indiana, USA" will be given instead.
Arguably that's incorrect, but it is the behaviour I want.
If "exact" is not given,
it will go on to look just for the town if the street isn't found.

The word "county" is removed from US county searches,
that either C<Leesburg, Loudoun County, Virginia, US> or C<Leesburg, Loudoun, Virginia, US> will work.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict) - exactly one of location or scantext is required
    location     => { type => 'scalar',   optional => 1 }  # address string (exclusive with scantext)
    scantext     => { type => 'scalar',   optional => 1 }  # free text to scan for place names
    region       => { type => 'scalar',   optional => 1 }  # ISO 3166-1 alpha-2 country code hint
    ignore_words => { type => 'arrayref', optional => 1 }  # words to suppress during scantext scan
    exact        => { type => 'scalar',   optional => 1 }  # if set, do not fall back to town-only match

=head4 output

    # Output schema (Return::Set)
    # scalar context: { type => 'object',   isa => 'Geo::Location::Point', optional => 1 }
    # list context:   { type => 'arrayref', of  => { isa => 'Geo::Location::Point' } }

=head4 provenance

Every result carries C<geocoder>, C<database>, C<confidence> and C<accuracy>
(section 2.1 of F<BI-Phase6-Dependency-Specs.md>).  This backend reports
C<database> as C<OpenAddresses>, and C<accuracy> is C<address> when the database
answered for the address itself and C<locality> when a hand-curated
C<%known_locations> entry did, since that entry's coordinate is a town's.

=cut
249
250sub geocode
251{
252
19
1
2227
        my $self = shift;
253
19
54
        my $param = (@_ == 1 && !ref($_[0]))
254                ? { location => $_[0] }
255                : Params::Get::get_params(undef, @_);
256
257
19
187
        my %ignore_words;
258
19
25
        if($param->{'ignore_words'}) {
259
0
0
0
0
0
0
                %ignore_words = map { lc($_) => 1 } @{$param->{'ignore_words'}};
260        }
261
262
19
22
        if(my $scantext = $param->{'scantext'}) {
263
4
11
                return if(length($scantext) < 6);
264                # FIXME:  wow this is inefficient
265
0
0
                $scantext =~ s/[^\w']+/ /g;
266
0
0
                my @words = split(/\s/, $scantext);
267
0
0
                my $count = scalar(@words);
268
0
0
                my $offset = 0;
269
0
0
                my @rc;
270
0
0
                my $region = $param->{'region'};
271
0
0
                if($region) {
272
0
0
                        $region = uc($region);
273                }
274
0
0
                while($offset < $count) {
275
0
0
                        if(length($words[$offset]) < 2) {
276
0
0
                                $offset++;
277
0
0
                                next;
278                        }
279
0
0
                        if(exists($ignore_words{lc($words[$offset])})) {
280
0
0
                                $offset++;
281
0
0
                                next;
282                        }
283
0
0
                        my $l;
284
0
0
                        if(($l = $self->geocode(location => $words[$offset])) && ref($l)) {
285
0
0
                                push @rc, $l;
286                        }
287
0
0
                        if($offset < $count - 1) {
288
0
0
                                my $addr = join(', ', $words[$offset], $words[$offset + 1]);
289
0
0
                                if(length($addr) == 0) {
290
0
0
                                        $offset++;
291                                }
292                                # https://stackoverflow.com/questions/11160192/how-to-parse-freeform-street-postal-address-out-of-text-and-into-components
293                                # TODO: Support longer addresses
294
0
0
                                if($addr =~ /\s+(?:\d{2,5}\s+)(?![ap]m\b)(?:(?:[a-zA-Z\s]{1,5}){1,2})?(?:[\s,.]+)?(?:(?:[a-zA-Z\s]{1,30}){1,4})(?:court|ct|street|st|drive|dr|lane|ln|road|rd|blvd)(?:[\s,.;]+)?(?:(?:[a-zA-Z\s]{1,30}){1,2})(?:[\s,.]+)?\b(?:AK|AL|AR|AZ|CA|CO|CT|DC|DE|FL|GA|GU|HI|IA|ID|IL|IN|KS|KY|LA|MA|MD|ME|MI|MN|MO|MS|MT|NC|ND|NE|NH|NJ|NM|NV|NY|OH|OK|OR|PA|RI|SC|SD|TN|TX|UT|VA|VI|VT|WA|WI|WV|WY)(?:[\s,.]+)?(?:\s+\d{5})?(?:[\s,.]+)/i) {
295
0
0
                                        unless($region && ($region ne 'US')) {
296
0
0
                                                if(($l = $self->geocode(location => "$addr, US")) && ref($l)) {
297
0
0
                                                        $l->confidence(0.8);
298
0
0
                                                        $l->country('US');
299
0
0
                                                        $l->location("$addr, USA");
300
0
0
                                                        push @rc, $l;
301                                                }
302                                        }
303                                } elsif($addr =~ /\s+(?:\d{2,5}\s+)(?![ap]m\b)(?:(?:[a-zA-Z\s]{1,5}){1,2})?(?:[\s,.]+)?(?:(?:[a-zA-Z\s]{1,30}){1,4})(?:court|ct|street|st|drive|dr|lane|ln|road|rd|blvd)(?:[\s,.;]+)?(?:(?:[a-zA-Z\s]{1,30}){1,2})(?:[\s,.]+)?\b(?:AB|BC|MB|NB|NL|NT|NS|ON|PE|QC|SK|YT)(?:[\s,.]+)?(?:\s+\d{5})?(?:[\s,.]+)/i) {
304
0
0
                                        unless($region && ($region ne 'CA')) {
305
0
0
                                                if(($l = $self->geocode(location => "$addr, Canada")) && ref($l)) {
306
0
0
                                                        $l->confidence(0.8);
307
0
0
                                                        $l->country('CA');
308
0
0
                                                        $l->location("$addr, Canada");
309
0
0
                                                        push @rc, $l;
310                                                }
311                                        }
312                                } elsif($addr =~ /([A-Za-z]+(?:\s+[A-Za-z]+)*)([\s,.]+)?\b(AK|AL|AR|AZ|CA|CO|CT|DC|DE|FL|GA|GU|HI|IA|ID|IL|IN|KS|KY|LA|MA|MD|ME|MI|MN|MO|MS|MT|NC|ND|NE|NH|NJ|NM|NV|NY|OH|OK|OR|PA|RI|SC|SD|TN|TX|UT|VA|VI|VT|WA|WI|WV|WY)/i) {
313
0
0
                                        unless($region && ($region ne 'US')) {
314
0
0
                                                if(($l = $self->geocode(location => "$addr, US")) && ref($l)) {
315
0
0
                                                        $l->confidence(0.6);
316
0
0
                                                        $l->city(uc($1));
317
0
0
                                                        $l->state(uc($3));
318
0
0
                                                        $l->country('US');
319
0
0
                                                        $l->location(uc("$addr, USA"));
320
0
0
                                                        push @rc, $l;
321                                                }
322                                        }
323                                } elsif($addr =~ /([A-Za-z]+(?:\s+[A-Za-z]+)*)([\s,.]+)?\b(AB|BC|MB|NB|NL|NT|NS|ON|PE|QC|SK|YT)/i) {
324
0
0
                                        unless($region && ($region ne 'CA')) {
325
0
0
                                                if(($l = $self->geocode(location => "$addr, Canada")) && ref($l)) {
326
0
0
                                                        $l->confidence(0.6);
327
0
0
                                                        $l->city(uc($1));
328
0
0
                                                        $l->state(uc($3));
329
0
0
                                                        $l->country('Canada');
330
0
0
                                                        $l->location(uc("$addr, Canada"));
331
0
0
                                                        push @rc, $l;
332                                                }
333                                        }
334                                }
335
0
0
                                if($region && (($l = $self->geocode(location => "$addr, $region")) && ref($l))) {
336
0
0
                                        $l->confidence(0.2);
337
0
0
                                        $l->location("$addr, $region");
338
0
0
                                        push @rc, $l;
339                                } elsif((!$region) && (($l = $self->geocode(location => $addr)) && ref($l))) {
340
0
0
                                        $l->confidence(0.1);
341
0
0
                                        $l->location($addr);
342
0
0
                                        push @rc, $l;
343                                }
344
0
0
                                if($offset < $count - 2) {
345
0
0
                                        $addr = join(', ', $words[$offset], $words[$offset + 1], $words[$offset + 2]);
346
0
0
                                        if(($l = $self->geocode(location => $addr)) && ref($l)) {
347
0
0
                                                $l->confidence(1.0);
348
0
0
                                                $l->location($addr);
349
0
0
                                                push @rc, $l;
350                                        }
351                                }
352                        }
353
0
0
                        $offset++;
354                }
355
0
0
                return @rc;
356                # my @locations;
357
358                # foreach my $l(@rc) {
359                        # push @locations, Location::GeoTool->create_coord($l->{'latitude'}, $l->{'longitude'}, $l->{'location'}, 'Degree');
360                # }
361
362                # return @locations;
363        }
364
365        my $location = $param->{location}
366
15
24
                or Carp::croak('Usage: geocode(location => $location|scantext => $text)');
367
368
369
14
24
        $location =~ s/,\s+,\s+/, /g;
370
371
14
16
        if($location =~ /^,\s*(.+)/) {
372
0
0
                $location = $1;
373        }
374
375        # Fail when the input is just a set of numbers.  The rule is the one
376        # Geo::Location::Predicate defines and the facade applies too, so what this
377        # backend refuses cannot drift away from what valid_location() answers.  As
378        # before, the answer is a miss rather than an error, because the caller may
379        # have another backend to try.
380
14
21
        return unless _valid_location($location);
381
11
49
        return if(length($location) <= 1);
382
383
11
327
        if($location =~ /^(.+),?\s*Washington\s*DC$/i) {
384
0
0
                $location = "$1, Washington, DC, USA";
385        } elsif($location =~ /^(.*),?\s*Saint Louis, (Missouri|MO)(.*)$/) {
386                # createdatabase.PL also maps this
387
0
0
                $location = "$1, St. Louis, MO$3";
388        }
389
390
11
19
        if(my $rc = $known_locations{$location}) {
391                # return $known_locations{$location};
392                return Geo::Location::Point->new({
393                        'lat' => $rc->{'latitude'},
394                        'long' => $rc->{'longitude'},
395
5
24
                        'lng' => $rc->{'longitude'},
396                        'location' => $location,
397                        'geocoder' => GEOCODER,
398                        'database' => 'OpenAddresses',
399                        # A hand-curated entry: the string matched it exactly, and the
400                        # coordinate it carries is a town's, not an address's.
401                        'confidence' => 1.0,
402                        'accuracy' => 'locality'
403                });
404        }
405
406
6
11
        $self->{'location'} = $location;
407
408
6
10
        my $county;
409        my $state;
410
6
0
        my $country;
411
6
0
        my $street;
412
413
6
5
        $location =~ s/\.//g;
414
415
6
15
        if($location !~ /,/) {
416
3
173
                if($location =~ /^(.+?)\s+(United States|USA|US)$/i) {
417
0
0
                        my $l = $1;
418
0
0
                        if(my $rc = $self->_get($l, 'US')) {
419
0
0
                                $rc->{'country'} = 'US';
420
0
0
                                return $rc;
421                        }
422
0
0
                        $l =~ s/\s+//g;
423
0
0
                        if(my $rc = $self->_get($l, 'US')) {
424
0
0
                                $rc->{'country'} = 'US';
425
0
0
                                return $rc;
426                        }
427                } elsif($location =~ /^(.+?)\s+(England|Scotland|Wales|Northern Ireland|UK|GB)$/i) {
428
0
0
                        my $l = $1;
429
0
0
                        $l =~ s/\s+//g;
430
0
0
                        if(my $rc = $self->_get($l, 'GB')) {
431
0
0
                                $rc->{'country'} = 'GB';
432
0
0
                                return $rc;
433                        }
434                } elsif($location =~ /^(.+?)\s+Canada$/i) {
435
0
0
                        my $l = $1;
436
0
0
                        $l =~ s/\s+//g;
437
0
0
                        if(my $rc = $self->_get($l, 'CA')) {
438
0
0
                                $rc->{'country'} = 'CA';
439
0
0
                                return $rc;
440                        }
441                }
442        }
443
6
5
        my $ap;
444
6
36
        if(($location =~ /USA$/) || ($location =~ /United States$/)) {
445
0
0
                $ap = $self->{'ap'}->{'us'} // Lingua::EN::AddressParse->new(country => 'US', auto_clean => 1, force_case => 1, force_post_code => 0);
446
0
0
                $self->{'ap'}->{'us'} = $ap;
447        } elsif($location =~ /(England|Scotland|Wales|Northern Ireland|UK|GB)$/i) {
448
0
0
                $ap = $self->{'ap'}->{'gb'} // Lingua::EN::AddressParse->new(country => 'GB', auto_clean => 1, force_case => 1, force_post_code => 0);
449
0
0
                $self->{'ap'}->{'gb'} = $ap;
450        } elsif($location =~ /Canada$/) {
451
0
0
                $ap = $self->{'ap'}->{'ca'} // Lingua::EN::AddressParse->new(country => 'CA', auto_clean => 1, force_case => 1, force_post_code => 0);
452
0
0
                $self->{'ap'}->{'ca'} = $ap;
453        } elsif($location =~ /Australia$/) {
454
0
0
                $ap = $self->{'ap'}->{'au'} // Lingua::EN::AddressParse->new(country => 'AU', auto_clean => 1, force_case => 1, force_post_code => 0);
455
0
0
                $self->{'ap'}->{'au'} = $ap;
456        }
457
6
6
        if($ap) {
458
0
0
                my $l = $location;
459
0
0
                if($l =~ /(.+), (England|UK)$/i) {
460
0
0
                        $l = "$1, GB";
461                }
462
0
0
                if($ap->parse($l)) {
463                        # Carp::croak($ap->report());
464                } else {
465
0
0
                        my %c = $ap->components();
466
0
0
                        my %addr = ( 'location' => $l );
467
0
0
                        $street = $c{'street_name'};
468
0
0
                        if(my $type = $c{'street_type'}) {
469
0
0
                                if(my $a = _abbreviate($type)) {
470
0
0
                                        $street .= " $a";
471                                } else {
472
0
0
                                        $street .= " $type";
473                                }
474
0
0
                                if(my $suffix = $c{'street_direction_suffix'}) {
475
0
0
                                        $street .= " $suffix";
476                                }
477
0
0
                                $street =~ s/^0+//;     # Turn 04th St into 4th St
478
0
0
                                $addr{'road'} = $street;
479                        }
480
0
0
                        if(length($c{'subcountry'}) == 2) {
481
0
0
                                $addr{'state'} = $c{'subcountry'};
482                        } else {
483
0
0
                                if($c{'country'} =~ /Canada/i) {
484
0
0
                                        $addr{'country'} = 'CA';
485
0
0
                                        if(my $twoletterstate = ($_locale_ca //= Locale::CA->new())->{province2code}{uc($c{'subcountry'})}) {
486
0
0
                                                $addr{'state'} = $twoletterstate;
487                                        }
488                                } elsif($c{'country'} =~ /^(United States|USA|US)$/i) {
489
0
0
                                        $addr{'country'} = 'US';
490
0
0
                                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($c{'subcountry'})}) {
491
0
0
                                                $addr{'state'} = $twoletterstate;
492                                        }
493                                } elsif($c{'country'}) {
494
0
0
                                        $addr{'country'} = $c{'country'};
495
0
0
                                        if($c{'subcountry'}) {
496
0
0
                                                $addr{'state'} = $c{'subcountry'};
497                                        }
498                                }
499                        }
500
0
0
                        $addr{'house_number'} = $c{'property_identifier'};
501
0
0
                        $addr{'city'} = $c{'suburb'};
502
0
0
                        if($addr{'house_number'}) {
503
0
0
                                if(my $rc = $self->_search(\%addr, ('house_number', 'road', 'city', 'state', 'country'))) {
504
0
0
                                        return $rc;
505                                }
506                        }
507
0
0
                        if((!$addr{'house_number'}) || !$param->{'exact'}) {
508
0
0
                                if(my $rc = $self->_search(\%addr, ('road', 'city', 'state', 'country'))) {
509
0
0
                                        return $rc;
510                                }
511                        }
512                }
513        }
514
515
6
118
        if($location =~ /^(.+?)[,\s]+(United States|USA|US)$/i) {
516                # Try Geo::StreetAddress::US, which is rather buggy
517
518
0
0
                my $l = $1;
519
0
0
                $l =~ s/,/ /g;
520
0
0
                $l =~ s/\s\s+/ /g;
521
522                # Work around for RT#122617
523
0
0
                if(($location !~ /\sCounty,/i) && (my $href = (Geo::StreetAddress::US->parse_location($l) || Geo::StreetAddress::US->parse_address($l)))) {
524
0
0
                        if($state = $href->{'state'}) {
525
0
0
                                if(length($state) > 2) {
526
0
0
                                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
527
0
0
                                                $state = $twoletterstate;
528                                        }
529                                }
530
0
0
                                my $city;
531
0
0
                                if($href->{city}) {
532
0
0
                                        $city = uc($href->{city});
533                                }
534
0
0
                                if($street = $href->{street}) {
535
0
0
                                        if($href->{'type'} && (my $type = _abbreviate($href->{'type'}))) {
536
0
0
                                                $street .= " $type";
537                                        }
538
0
0
                                        if($href->{suffix}) {
539
0
0
                                                $street .= ' ' . $href->{suffix};
540                                        }
541
0
0
                                        if(my $prefix = $href->{prefix}) {
542
0
0
                                                $street = "$prefix $street";
543                                        }
544
0
0
                                        if($href->{'number'}) {
545
0
0
                                                if(my $rc = $self->_get($href->{'number'}, "$street$city$state", 'US')) {
546
0
0
                                                        $rc->{'country'} = 'US';
547
0
0
                                                        return $rc;
548                                                }
549                                        }
550
0
0
                                        if(my $rc = $self->_get("$street$city$state", 'US')) {
551
0
0
                                                $rc->{'country'} = 'US';
552
0
0
                                                return $rc;
553                                        }
554                                }
555                        }
556                }
557
558                # Hack to find "name, street, town, state, US"
559
0
0
                my @addr = split(/,\s*/, $location);
560
0
0
                if(scalar(@addr) == 5) {
561
0
0
                        $state = $addr[3];
562
0
0
                        if(length($state) > 2) {
563
0
0
                                if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
564
0
0
                                        $state = $twoletterstate;
565                                }
566                        }
567
0
0
                        if(length($state) == 2) {
568
0
0
                                $addr[1] = _normalize($addr[1]);
569
0
0
                                if(my $rc = $self->_get($addr[0], $addr[1], $addr[2], $state, 'US')) {
570
0
0
                                        $rc->{'country'} = 'US';
571
0
0
                                        return $rc;
572                                }
573                        }
574                        # Hack to find "street, town, county, state, US"
575
0
0
                        if(length($state) == 2) {
576
0
0
                                $addr[0] = _normalize($addr[0]);
577
0
0
                                $addr[2] =~ s/\s+COUNTY$//i;
578
0
0
                                if(my $rc = $self->_get($addr[0], $addr[1], $addr[2], $state, 'US')) {
579
0
0
                                        $rc->{'country'} = 'US';
580
0
0
                                        return $rc;
581                                }
582
0
0
                                if(my $rc = $self->_get($addr[0], $addr[1], $state, 'US')) {
583
0
0
                                        $rc->{'country'} = 'US';
584
0
0
                                        return $rc;
585                                }
586                        }
587                }
588        }
589
590
6
126
        if($location =~ /(.+),\s*([\s\w]+),\s*([\w\s]+)$/) {
591
3
4
                my $city = $1;
592
3
4
                $state = $2;
593
3
3
                $country = $3;
594
3
4
                $state =~ s/\s$//g;
595
3
3
                $country =~ s/\s$//g;
596
597
3
2
                my $c;
598
599
3
13
                if((uc($country) eq 'ENGLAND') ||
600                   (uc($country) eq 'SCOTLAND') ||
601                   (uc($country) eq 'WALES')) {
602
0
0
                        $country = 'Great Britain';
603
0
0
                        $c = 'gb';
604                } else {
605
3
6
                        $c = country2code($country);
606                }
607
3
68
                if($c) {
608
0
0
                        if($c eq 'us') {
609
0
0
                                if(length($state) > 2) {
610
0
0
                                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
611
0
0
                                                $state = $twoletterstate;
612                                        }
613                                }
614
0
0
                                my $rc;
615
616
0
0
                                if($city !~ /,/) {
617
0
0
                                        $city = uc($city);
618
0
0
                                        if($city =~ /^(.+)\sCOUNTY$/) {
619                                                # Simple case looking up a county in a state in the US
620
0
0
                                                if($rc = $self->_get("$1$state", 'US')) {
621
0
0
                                                        $rc->{'country'} = 'US';
622
0
0
                                                        return $rc;
623                                                }
624                                        } else {
625                                                # Simple case looking up a city in a state in the US
626
0
0
                                                if($rc = $self->_get("$city$state", 'US')) {
627
0
0
                                                        $rc->{'country'} = 'US';
628
0
0
                                                        return $rc;
629                                                }
630                                        }
631                                } elsif(my $href = Geo::StreetAddress::US->parse_address("$city, $state")) {
632                                # warn __LINE__;
633                                        # Well formed, simple street address in the US
634
0
0
                                        $state = $href->{'state'};
635
0
0
                                        if(length($state) > 2) {
636
0
0
                                                if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
637
0
0
                                                        $state = $twoletterstate;
638                                                }
639                                        }
640
0
0
                                        if($href->{city}) {
641
0
0
                                                $city = uc($href->{city});
642                                        }
643                                        # Unabbreviated - look up both, helps with fallback to Maxmind
644
0
0
                                        my $fullstreet = $href->{'street'};
645                                # warn __LINE__;
646
0
0
                                        if($street = $fullstreet) {
647                                # warn __LINE__;
648
0
0
                                                $fullstreet .= ' ' . $href->{'type'};
649
0
0
                                                if(my $type = _abbreviate($href->{'type'})) {
650
0
0
                                                        $street .= " $type";
651                                                }
652
0
0
                                                if($href->{suffix}) {
653
0
0
                                                        $street .= ' ' . $href->{suffix};
654
0
0
                                                        $fullstreet .= ' ' . $href->{suffix};
655                                                }
656                                        }
657
0
0
                                        if($street) {
658                                # warn __LINE__;
659
0
0
                                                if(my $prefix = $href->{prefix}) {
660
0
0
                                                        $street = "$prefix $street";
661
0
0
                                                        $fullstreet = "$prefix $fullstreet";
662                                                }
663
0
0
                                                if($href->{'number'}) {
664
0
0
                                                        if($rc = $self->_get($href->{'number'}, "$street$city$state", 'US')) {
665
0
0
                                                                $rc->{'country'} = 'US';
666
0
0
                                                                return $rc;
667                                                        }
668
0
0
                                                        if($rc = $self->_get($href->{'number'}, "$fullstreet$city$state", 'US')) {
669
0
0
                                                                $rc->{'country'} = 'US';
670
0
0
                                                                return $rc;
671                                                        }
672                                                }
673                                                # warn("$street$city$state", 'US');
674
0
0
                                                if($rc = $self->_get("$street$city$state", 'US')) {
675
0
0
                                                        $rc->{'country'} = 'US';
676
0
0
                                                        return $rc;
677                                                }
678
0
0
                                                $street =~ s/\s+//g;
679
0
0
                                                if($rc = $self->_get("$street$city$state", 'US')) {
680
0
0
                                                        $rc->{'country'} = 'US';
681
0
0
                                                        return $rc;
682                                                }
683                                                # warn("$fullstreet$city$state", 'US');
684
0
0
                                                if($rc = $self->_get("$fullstreet$city$state", 'US')) {
685
0
0
                                                        $rc->{'country'} = 'US';
686
0
0
                                                        return $rc;
687                                                }
688
0
0
                                                $fullstreet =~ s/\s+//g;
689                                        }
690
0
0
                                        Carp::carp("Fast lookup of US location '$location' failed");
691                                } else {
692
0
0
                                        if($city =~ /^(\d.+),\s*([\w\s]+),\s*([\w\s]+)/) {
693
0
0
                                                my $lookup = "$1, $2, $state";
694
0
0
                                                if(my $href = (Geo::StreetAddress::US->parse_address($lookup) || Geo::StreetAddress::US->parse_location($lookup))) {
695                                                        # Street, City, County
696                                                        # 105 S. West Street, Spencer, Owen, Indiana, USA
697
0
0
                                                        $county = $3;
698
0
0
                                                        $county =~ s/\s*county$//i;
699
0
0
                                                        if($href->{'state'}) {
700
0
0
                                                                $state = $href->{'state'};
701                                                        } else {
702
0
0
                                                                Carp::croak(__PACKAGE__, ": Geo::StreetAddress::US couldn't find the state in '$lookup'");
703                                                        }
704
0
0
                                                        if(length($state) > 2) {
705
0
0
                                                                if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
706
0
0
                                                                        $state = $twoletterstate;
707                                                                }
708                                                        }
709
0
0
                                                        my %args = (county => uc($county), state => $state, country => 'US');
710
0
0
                                                        if($href->{city}) {
711
0
0
                                                                $city = $args{city} = uc($href->{city});
712                                                        }
713
0
0
                                                        if($href->{number}) {
714
0
0
                                                                $args{number} = $href->{number};
715                                                        }
716
0
0
                                                        if($street = $href->{street}) {
717
0
0
                                                                if(my $type = _abbreviate($href->{'type'})) {
718
0
0
                                                                        $street .= " $type";
719                                                                }
720
0
0
                                                                if($href->{suffix}) {
721
0
0
                                                                        $street .= ' ' . $href->{suffix};
722                                                                }
723
0
0
                                                                if(my $prefix = $href->{prefix}) {
724
0
0
                                                                        $street = "$prefix $street";
725                                                                }
726
0
0
                                                                $args{street} = uc($street);
727
0
0
                                                                if($href->{'number'}) {
728
0
0
                                                                        if($county) {
729
0
0
                                                                                if($rc = $self->_get($href->{'number'}, "$street$city$county$state", 'US')) {
730
0
0
                                                                                        $rc->{'country'} = 'US';
731
0
0
                                                                                        return $rc;
732                                                                                }
733                                                                        }
734
0
0
                                                                        if($rc = $self->_get($href->{'number'}, "$street$city$state", 'US')) {
735
0
0
                                                                                $rc->{'country'} = 'US';
736
0
0
                                                                                return $rc;
737                                                                        }
738
0
0
                                                                        if($county) {
739
0
0
                                                                                if($rc = $self->_get("$street$city$county$state", 'US')) {
740
0
0
                                                                                        $rc->{'country'} = 'US';
741
0
0
                                                                                        return $rc;
742                                                                                }
743                                                                        }
744
0
0
                                                                        if($rc = $self->_get("$street$city$state", 'US')) {
745
0
0
                                                                                $rc->{'country'} = 'US';
746
0
0
                                                                                return $rc;
747                                                                        }
748                                                                }
749                                                        }
750
0
0
                                                        return; # Not found
751                                                }
752
0
0
                                                Carp::croak($city);     # TODO: do something here
753                                        } elsif($city =~ /^(\w[\w\s]+),\s*([\w\s]+)/) {
754                                                # Perhaps it just has the street's name?
755                                                # Rockville Pike, Rockville, MD, USA
756
0
0
                                                my $first = uc($1);
757
0
0
                                                my $second = uc($2);
758
0
0
                                                if($second =~ /(\d+)\s+(.+)/) {
759
0
0
                                                        $second = "$1$2";
760                                                }
761
0
0
                                                if($rc = $self->_get("$first$second$state", 'US')) {
762
0
0
                                                        $rc->{'country'} = 'US';
763
0
0
                                                        return $rc;
764                                                }
765                                                # Perhaps it's a city in a county?
766                                                # Silver Spring, Montgomery County, MD, USA
767
0
0
                                                $second =~ s/\s+COUNTY$//;
768
0
0
                                                if($rc = $self->_get("$first$second$state", 'US')) {
769
0
0
                                                        $rc->{'country'} = 'US';
770
0
0
                                                        return $rc;
771                                                }
772                                                # Not all the database has the county
773
0
0
                                                if($rc = $self->_get("$first$state", 'US')) {
774
0
0
                                                        $rc->{'country'} = 'US';
775
0
0
                                                        return $rc;
776                                                }
777                                                # Brute force last ditch approach
778
0
0
                                                my $copy = uc($location);
779
0
0
                                                $copy =~ s/,\s+//g;
780
0
0
                                                $copy =~ s/\s*USA$//;
781
0
0
                                                if($rc = $self->_get($copy, 'US')) {
782
0
0
                                                        $rc->{'country'} = 'US';
783
0
0
                                                        return $rc;
784                                                }
785
0
0
                                                if($copy =~ s/(\d+)\s+/$1/) {
786
0
0
                                                        if($rc = $self->_get($copy, 'US')) {
787
0
0
                                                                $rc->{'country'} = 'US';
788
0
0
                                                                return $rc;
789                                                        }
790                                                }
791                                        }
792                                        # warn "Can't yet parse US location '$location'";
793                                }
794                        } elsif($c eq 'ca') {
795
0
0
                                if(length($state) > 2) {
796
0
0
                                        if(my $twoletterstate = ($_locale_ca //= Locale::CA->new())->{province2code}{uc($state)}) {
797
0
0
                                                $state = $twoletterstate;
798                                        }
799                                }
800
0
0
                                my $rc;
801
0
0
                                if($city !~ /,/) {
802                                        # Simple case looking up a city in a state in Canada
803
0
0
                                        $city = uc($city);
804
0
0
                                        if($rc = $self->_get("$city$state", 'CA')) {
805
0
0
                                                $rc->{'country'} = 'CA';
806
0
0
                                                return $rc;
807                                        }
808                                # } elsif(my $href = Geo::StreetAddress::Canada->parse_address("$city, $state")) {
809                                } elsif(my $href = 0) {
810                                        # Well formed, simple street address in Canada
811
0
0
                                        $state = $href->{'province'};
812
0
0
                                        if(length($state) > 2) {
813
0
0
                                                if(my $twoletterstate = ($_locale_ca //= Locale::CA->new())->{province2code}{uc($state)}) {
814
0
0
                                                        $state = $twoletterstate;
815                                                }
816                                        }
817
0
0
                                        my %args = (state => $state, country => 'CA');
818
0
0
                                        if($href->{city}) {
819
0
0
                                                $args{city} = uc($href->{city});
820                                        }
821
0
0
                                        if($href->{number}) {
822
0
0
                                                $args{number} = $href->{number};
823                                        }
824
0
0
                                        if($street = $href->{street}) {
825
0
0
                                                if(my $type = _abbreviate($href->{'type'})) {
826
0
0
                                                        $street .= " $type";
827                                                }
828
0
0
                                                if($href->{suffix}) {
829
0
0
                                                        $street .= ' ' . $href->{suffix};
830                                                }
831                                        }
832
0
0
                                        if($street) {
833
0
0
                                                if(my $prefix = $href->{prefix}) {
834
0
0
                                                        $street = "$prefix $street";
835                                                }
836
0
0
                                                $args{street} = uc($street);
837                                        }
838
0
0
                                        Carp::carp("Fast lookup of Canadian location '$location' failed");
839                                } else {
840
0
0
                                        if($city =~ /^(\w[\w\s]+),\s*([\w\s]+)/) {
841                                                # Perhaps it just has the street's name?
842                                                # Rockville Pike, Rockville, MD, USA
843
0
0
                                                my $first = uc($1);
844
0
0
                                                my $second = uc($2);
845
0
0
                                                if($rc = $self->_get("$first$second$state", 'CA')) {
846
0
0
                                                        $rc->{'country'} = 'CA';
847
0
0
                                                        return $rc;
848                                                }
849                                                # Perhaps it's a city in a county?
850                                                # Silver Spring, Montgomery County, MD, USA
851
0
0
                                                $second =~ s/\s+COUNTY$//;
852
0
0
                                                if($rc = $self->_get("$first$second$state", 'CA')) {
853
0
0
                                                        $rc->{'country'} = 'CA';
854
0
0
                                                        return $rc;
855                                                }
856                                                # Not all the database has the county
857
0
0
                                                if($rc = $self->_get("$first$state", 'CA')) {
858
0
0
                                                        $rc->{'country'} = 'CA';
859
0
0
                                                        return $rc;
860                                                }
861                                                # Brute force last ditch approach
862
0
0
                                                my $copy = uc($location);
863
0
0
                                                $copy =~ s/,\s+//g;
864
0
0
                                                $copy =~ s/\s*Canada$//i;
865
0
0
                                                if($rc = $self->_get($copy, 'CA')) {
866
0
0
                                                        $rc->{'country'} = 'CA';
867
0
0
                                                        return $rc;
868                                                }
869
0
0
                                                if($copy =~ s/(\d+)\s+/$1/) {
870
0
0
                                                        if($rc = $self->_get($copy, 'CA')) {
871
0
0
                                                                $rc->{'country'} = 'CA';
872
0
0
                                                                return $rc;
873                                                        }
874                                                }
875                                        }
876                                        # warn "Can't yet parse Canadian location '$location'";
877                                }
878                        } else {
879                                # Currently only handles Town, Region, Country
880                                # TODO: add addresses support
881
0
0
                                if(($c eq 'au') && (length($state) > 3)) {
882
0
0
                                        if(my $abbrev = Locale::SubCountry->new('AU')->code(ucfirst(lc($state)))) {
883
0
0
                                                if($abbrev ne 'unknown') {
884
0
0
                                                        $state = $abbrev;
885                                                }
886                                        }
887                                }
888
0
0
                                if($city =~ /^(\w[\w\s]+),\s*([,\w\s]+)/) {
889                                        # City includes a street name
890
0
0
                                        $street = uc($1);
891
0
0
                                        $city = uc($2);
892
0
0
                                        my $number;
893
0
0
                                        if($street =~ /^(\d+)\s+(.+)/) {
894
0
0
                                                $number = $1;
895
0
0
                                                $street = $2;
896                                        }
897
898                                        # TODO: Configurable - or better still remove the need
899
0
0
                                        if($city eq 'MINSTER, THANET') {
900
0
0
                                                $city = 'RAMSGATE';
901                                        }
902
0
0
                                        $street = _normalize($street);
903
0
0
                                        if($number) {
904
0
0
                                                if(my $rc = $self->_get("$number$street$city$state$c")) {
905
0
0
                                                        return $rc;
906                                                }
907                                                # If we can't find the number, at least find the road
908                                        }
909
0
0
                                        if(my $rc = $self->_get("$street$city$state$c")) {
910
0
0
                                                return $rc;
911                                        }
912                                }
913
0
0
                                if((!$street) || !$param->{'exact'}) {
914
0
0
                                        if(my $rc = $self->_get("$city$state$c")) {
915                                                # return {
916                                                        # 'number' => undef,
917                                                        # 'street' => undef,
918                                                        # 'city' => $city,
919                                                        # 'state' => $state,
920                                                        # 'country' => $country,
921                                                        # %{$rc}
922                                                # };
923
0
0
                                                return $rc;
924                                        }
925                                }
926                        }
927                }
928        } elsif($location =~ /([a-z\s]+),?\s*(United States|USA|US|Canada)$/i) {
929                # Looking for a state/province in Canada or the US
930
0
0
                $state = $1;
931
0
0
                $country = $2;
932
0
0
                if($country =~ /Canada/i) {
933
0
0
                        $country = 'CA';
934
0
0
                        if(length($state) > 2) {
935
0
0
                                if(my $twoletterstate = ($_locale_ca //= Locale::CA->new())->{province2code}{uc($state)}) {
936
0
0
                                        $state = $twoletterstate;
937                                }
938                        }
939                } else {
940
0
0
                        $country = 'US';
941
0
0
                        if(length($state) > 2) {
942
0
0
                                if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($state)}) {
943
0
0
                                        $state = $twoletterstate;
944                                }
945                        }
946                }
947
0
0
                if(my $rc = $self->_get("$state$country")) {
948
0
0
                        $rc->{'country'} = $country;
949
0
0
                        return $rc;
950                }
951        }
952
953
6
7
        if($country) {
954                # Geo::Address::Parser is optional; skip this step if it isn't installed
955
3
15
                unless(Geo::Address::Parser->can('parse')) {
956
3
3
0
3
77
0
                        eval { require Geo::Address::Parser; Geo::Address::Parser->import() };
957                }
958
959
3
323
                if(($country eq 'US') && Geo::Address::Parser->can('parse')) {
960
0
0
                        my $addr_parser = Geo::Address::Parser->new(country => 'US');
961
0
0
                        if(my $fields = $addr_parser->parse($location)) {
962
0
0
0
0
                                for my $key (keys %{$fields}) {
963
0
0
                                        delete $fields->{$key} unless defined $fields->{$key};
964                                }
965
0
0
0
0
                                if(my $rc = $self->_search($fields, keys %{$fields})) {
966
0
0
                                        return $rc;
967                                }
968                        }
969                }
970        }
971
972        # Finally try libpostal,
973        # which is good but uses a lot of memory
974
6
8
        my $use_libpostal = $self->{'libpostal'} // 1;
975
6
7
        if($use_libpostal && ($libpostal_is_installed == LIBPOSTAL_UNKNOWN)) {
976
0
0
0
0
                if(eval { require Geo::libpostal; } ) {
977
0
0
                        Geo::libpostal->import();
978
0
0
                        $libpostal_is_installed = LIBPOSTAL_INSTALLED;
979                } else {
980
0
0
                        $libpostal_is_installed = LIBPOSTAL_NOT_INSTALLED;
981                }
982        }
983
984        # print(__PACKAGE__, ': ', __LINE__, ": libpostal_is_installed = $libpostal_is_installed ($location)\n");
985
986
6
7
        if($use_libpostal && ($libpostal_is_installed == LIBPOSTAL_INSTALLED) && (my %addr = Geo::libpostal::parse_address($location))) {
987
0
0
                if($addr{'country'} && $addr{'state'} && ($addr{'country'} =~ /^(Canada|United States|USA|US)$/i)) {
988
0
0
                        if($street = $addr{'road'}) {
989
0
0
                                $street = _normalize($street);
990
0
0
                                $addr{'road'} = $street;
991                        }
992
0
0
                        if($addr{'country'} =~ /Canada/i) {
993
0
0
                                $addr{'country'} = 'Canada';
994
0
0
                                if(length($addr{'state'}) > 2) {
995
0
0
                                        if(my $twoletterstate = ($_locale_ca //= Locale::CA->new())->{province2code}{uc($addr{'state'})}) {
996
0
0
                                                $addr{'state'} = $twoletterstate;
997                                        }
998                                }
999                        } else {
1000
0
0
                                $addr{'country'} = 'US';
1001
0
0
                                if(length($addr{'state'}) > 2) {
1002
0
0
                                        if(my $twoletterstate = ($_locale_us //= Locale::US->new())->{state2code}{uc($addr{'state'})}) {
1003
0
0
                                                $addr{'state'} = $twoletterstate;
1004                                        }
1005                                }
1006                        }
1007
0
0
                        if($addr{'state_district'}) {
1008
0
0
                                $addr{'state_district'} =~ s/^(.+)\s+COUNTY/$1/i;
1009
0
0
                                if(my $rc = $self->_search(\%addr, ('house_number', 'road', 'city', 'state_district', 'state', 'country'))) {
1010
0
0
                                        return $rc;
1011                                }
1012                        }
1013
0
0
                        if(my $rc = $self->_search(\%addr, ('house_number', 'road', 'city', 'state', 'country'))) {
1014
0
0
                                return $rc;
1015                        }
1016
0
0
                        if($addr{'house_number'}) {
1017
0
0
                                if(my $rc = $self->_search(\%addr, ('road', 'city', 'state', 'country'))) {
1018
0
0
                                        return $rc;
1019                                }
1020                        }
1021                }
1022        }
1023
6
9
        if($location =~ s/,//g) {
1024
3
28
                return $self->geocode($location);
1025        }
1026
3
10
        undef;
1027}
1028
1029# $data is a hashref to data such as returned by Geo::libpostal::parse_address
1030# @columns is the key names to use in $data
1031sub _search {
1032
0
0
        my ($self, $data, @columns) = @_;
1033
1034
0
0
        my $location;
1035
0
0
        foreach my $column(@columns) {
1036
0
0
                if($data->{$column}) {
1037
0
0
                        $location .= $data->{$column};
1038                }
1039        }
1040
0
0
        if($location) {
1041
0
0
                return $self->_get($location);
1042        }
1043}
1044
1045# State must be the abbreviated form
1046sub _miss_scope {
1047
7
7
        my $self = shift;
1048
1049
7
14
        return Geo::Coder::Free::MissCache->scope_for('OpenAddresses', $self->{'openaddr'} // '');
1050}
1051
1052sub _get {
1053
8
2703
        my ($self, @location) = @_;
1054
1055
8
10
        my $location = join('', @location);
1056
8
10
        $location =~ s/^\s+//;
1057
8
25
        $location =~ s/,\s*//g;
1058
8
8
        $location =~ tr/ž/z/;     # Remove wide characters
1059
8
7
        $location =~ s/\xc5\xbe/z/g;
1060
8
12
        $location =~ s/\N{U+017E}/z/g;
1061
8
22
        $location =~ s/\s+//g;
1062
1063
8
6
        my $digest;
1064
8
10
        if(length($location) <= 16) {
1065
2
2
                $digest = uc($location);
1066        } else {
1067
6
17
                $digest = substr Digest::MD5::md5_base64(uc($location)), 0, 16;
1068        }
1069
1070        # print __PACKAGE__, ': ', __LINE__, ': ', uc($location), " = $digest\n";
1071        # my @call_details = caller(0);
1072        # print "\t", ' called from line ', $call_details[2], "\n";
1073        # @call_details = caller(1);
1074        # print "\t", ' called from line ', $call_details[2], "\n";
1075
1076        # my @call_details = caller(0);
1077        # print "line ", $call_details[2], "\n";
1078        # print "$location: $digest\n";
1079
8
12
        if(my $cache = $self->{'cache'}) {
1080
2
5
                if(my $rc = $cache->get_object($digest)) {
1081
1
52
                        return Storable::thaw($rc->value());
1082                }
1083        }
1084
1085        # A miss is remembered for the whole process, and per data set: another
1086        # directory is another database and says nothing about this one.
1087
7
37
        my $miss_scope = $self->_miss_scope();
1088
7
8
        return if Geo::Coder::Free::MissCache->missed($miss_scope, $digest);
1089        my $openaddr_db = $self->{openaddr_db} ||
1090                Geo::Coder::Free::DB::openaddresses->new(
1091                        cache => $self->{cache} || CHI->new(driver => 'Memory', datastore => {}),
1092                        directory => $self->{openaddr},
1093
5
9
                        id => 'md5',
1094                        no_entry => 1,
1095                );
1096
5
5
        $self->{openaddr_db} = $openaddr_db;
1097
5
6
        $self->{'reads'}++;
1098
5
7
        if(my $geohash = $openaddr_db->geohash(md5 => $digest)) {
1099
3
53
                $self->{'geo_hash'} ||= Geo::Hash->new();
1100
3
13
                my ($latitude, $longitude) = $self->{'geo_hash'}->decode($geohash);
1101
1102
3
330
                my $rc = Geo::Location::Point->new({
1103                        'lat' => $latitude,
1104                        'long' => $longitude,
1105                        'lng' => $longitude,
1106                        'location' => $location,
1107                        'geocoder' => GEOCODER,
1108                        'database' => 'OpenAddresses',
1109                        # The database answered for the md5 of the normalised address, so the
1110                        # string matched exactly and the geohash is the address's position.
1111                        'confidence' => 1.0,
1112                        'accuracy' => 'address'
1113                });
1114
1115
3
69
                if(my $cache = $self->{'cache'}) {
1116
1
2
                        $cache->set($digest, Storable::freeze($rc), '1 month');
1117                }
1118
1119
3
266
                return $rc;
1120        }
1121
2
8
        Geo::Coder::Free::MissCache->remember($miss_scope, $digest);
1122
2
4
        return;
1123}
1124
1125 - 1145
=head2  reverse_geocode

    $location = $geocoder->reverse_geocode(latlng => '37.778907,-122.39732');

To be done.  The database holds a hash of each address rather than the address
itself, so a coordinate cannot be turned back into one.  It always croaks
C<Reverse lookup is not yet supported>, whatever it is given.

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    # any arguments are accepted and ignored

=head4 output

    # Output schema (Return::Set)
    # none: always croaks

=cut
1146
1147# At the moment this can't be supported as the DB only has a hash in it
1148sub reverse_geocode {
1149
3
1
629
        Carp::croak(__PACKAGE__, ': Reverse lookup is not yet supported');
1150}
1151
1152 - 1168
=head2  ua

Does nothing, here for compatibility with other geocoders

=head3 API SPECIFICATION

=head4 input

    # Input schema (Params::Validate::Strict)
    # any arguments are accepted and ignored

=head4 output

    # Output schema (Return::Set)
    # nothing: an empty list, or undef in scalar context

=cut
1169
1170
1
sub ua {
1171}
1172
1173 - 1231
=head1 AUTHOR

Nigel Horne <njh@bandsman.co.uk>

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.

The contents of lib/Geo/Coder/Free/OpenAddresses/databases comes from
the places listed in the synopsis.

=head1 BUGS

Lots of lookups fail at the moment.

There are die()s where the code path has yet to be written.

The openaddresses data doesn't cover the globe.

Can't parse and handle "London, England".

Currently only searches US and Canadian data.

If you do search in the UK, only look up towns, full addresses aren't
included.  So these will print the same.

    use Geo::Coder::Free::OpenAddresses;

    $location = $geo_coder->geocode(location => '22 Central Road, Ramsgate, Kent, England');
    print $location->{latitude}, "\n";
    print $location->{longitude}, "\n";
    $location = $geo_coder->geocode(location => '7 Hillbrow Road, St Lawrence, Thanet, Kent, England');
    print $location->{latitude}, "\n";
    print $location->{longitude}, "\n";

When I added the WhosOnFirst data I should have renamed this as it contains
data from both sources.

The database shouldn't be called $OPENADDR_HOME/openaddresses.sql,
since the database now also includes data from WhosOnFirst.

The name openaddresses.sql shouldn't be hardcoded,
add support to "new" for the parameter "dbname".

The argument "openaddr",
would be less confusing if it were called "directory",

=head1 SEE ALSO

VWF, openaddresses.

=head1 LICENSE AND COPYRIGHT

Copyright 2017-2025 Nigel Horne.

The program code is released under the following licence: GPL for personal use on a single computer.
All other users (including Commercial, Charity, Educational, Government)
must apply in writing for a licence for use from Nigel Horne at `<njh at nigelhorne.com>`.

=cut
1232
12331;