Brad's Blog: Perl Archives

Tue Feb 17 17:07:06 GMT 2009

Finance::Quote::currency_lookup in action

Having written Finance::Quote::currency_lookup I was able to get back to the little script that triggered the idea in the first place

The idea: a script which I can call with partial code or currency names and an amount which will then try and tell me what the current converted value is. If there's something other than a single matching currency let me know that nothing or too many things matched.

Here's the script (also attached):

 1  #!/usr/bin/env perl
 2
 3  use strict;
 4  use warnings;
 5
 6  use Finance::Quote;
 7  my $q = Finance::Quote->new();
 8
 9  my %trans = ();
10  @trans{qw(from to)} = @ARGV;
11
12  # Copied this patter from Finance::Quote::currency to be consistent
13  $trans{from} =~ s/^\s*(\d*\.?\d*)\s*//;
14  my $amount = $1 || 1;
15
16  # Search out currencies
17  for my $key (qw( from to )) {
18    # Is it a code or a name?
19    my $currencies;
20    if ( $trans{$key} =~ /^[A-Z]{2,}$/ ) {
21      # Guess code
22      $currencies = $q->currency_lookup( code => qr/$trans{$key}/ );
23    }
24    else {
25      # Guess name
26      $currencies = $q->currency_lookup( name => qr/$trans{$key}/i );
27    }
28    if ( scalar keys %{$currencies} == 1 ) {
29      my $real_key = "real_${key}";
30      ($trans{$real_key}) = keys %{$currencies};
31      printf "%s : %s (%s)\n", $key
32                             , $trans{$real_key}
33                             , $currencies->{$trans{$real_key}}->{name};
34    }
35    elsif ( scalar keys %{$currencies} > 1 ) {
36      print "Multiple currency matches for ${key}:\n";
37      print " * $_ (" . $currencies->{$_}->{name} . ")\n" for keys %{$currencies};
38    }
39    else {
40      print "No currency matches for ${key}: " . $trans{$key} . "!\n";
41    }
42  }
43
44  # If real from and to exist run the conversion
45  if ( exists $trans{real_from} && exists $trans{real_to} ) {
46    print "${amount} $trans{real_from} => "
47        . $q->currency("${amount}$trans{real_from}", $trans{real_to})
48        . " $trans{real_to}\n";
49  }

And here's some examples of this script in action:

# Some Australian dollars to pounds
# (name searches only)
$ ./conv.pl 42aus brit
from : AUD (Australian Dollar)
to : GBP (British Pound)
42 AUD => 18.8454 GBP

# And the some pounds to US dollars
# (a name and a code search, because US is all-caps)
$ ./conv.pl brit US
from : GBP (British Pound)
to : USD (U.S. Dollar)
1 GBP => 1.4225 USD

# Oops - a typo on pounds
$ ./conv.pl GPB euro
No currency matches for from: GPB!
to : EUR (Euro)

# And now, to make the British cry...
$ ./conv.pl GBP euro
from : GBP (British Pound)
to : EUR (Euro)
1 GBP => 1.1317 EUR

# And finally, too many matches:
$ ./conv.pl egypt ven
from : EGP (Egyptian Pound)
Multiple currency matches for to:
 * SIT (Slovenian Tolar)
 * VEB (Venezuelan Bolivar)

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Tue Feb 17 01:52:12 GMT 2009

Adding Finance::Quote::currency_lookup to Finance::Quote

While using Finance::Quote::currency() to do some conversions I found myself having to go searching around the web for currency codes. At the time I thought it would be much easier if you could just asked the module what currency codes existed - and so I started work on a currency_lookup() function to do just that.

Finance::Quote uses the Yahoo Currency Converter website so the new function needed to report a compatible list of currencies.

Having developed the search method against a statically stored list of currencies (as I hadn't implemented the live extraction yet) I then discussed options with the other developers and decided that a hard coded list was a better approach given that the list wasn't going to change very often.

Here's a basic idea of usage:

1    $currencies_by_name = $q->currency_lookup( name => 'Australian' );
2    $currencies_by_code = $q->currency_lookup( code => qr/^b/i      );
3    $currencies_by_both = $q->currency_lookup( name => qr/pound/i
4                                             , code => 'GB'         );

The return value is a hash of hash-refs - for example for the first query above:

1    $VAR1 = { AUD => { name => "Australian Dollar" } }

I had added a module to contain the currency list (Finance::Quote::Currencies). Once the search code was complete I then added a test to compare the static list with a live list and implemented Finance::Quote::Currencies::fetch_live_currencies(). This function uses HTML::Parser to extract the currency list. If the currency list changes this function can be used to easily updated the stored list.

These changes are currently available on the currency_lookup branch on github. Coming to a CPAN release in your town real soon now.

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Thu Mar 27 00:59:12 GMT 2008

LWP::UserAgent fails with '500 Line too long (limit is 4096)'

This one caught somewhat by surprise - mostly because of the unusual use of a 500 (Internal Server Error) code by LWP::UserAgent to flag a client-side configuration problem.

The answer, as with so many things perl, lay with the PerlMonks in this article: Mysterious LWP status 500 Line too long.

To summarise: Net::HTTP has a default maximum header line length of 4096. This can be changed by passing a MaxLineLength option to the constructor - and happily LWP::UserAgent has an internal hash which can be tweaked to set the desired maximum line length:

1  my %OPTS = @LWP::Protocol::http::EXTRA_SOCK_OPTS;
2  $OPTS{MaxLineLength} = 8192; # Or however large is needed...
3  @LWP::Protocol::http::EXTRA_SOCK_OPTS = %OPTS;

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Thursday 14th September 15:49:36 AEST

PlainOldDocumentation Cheat-Sheet

There's a couple of cheat/reference sheets out there for perl already:

Here's a perlpod quick reference:

SECTIONS    INDENTATION         START/STOP POD
=head1      =over indentlevel   =pod
=head2      =item bullet        =cut
=head3      =back

FORMATTING CODES              FORMATTING CODES (cont.)
I     italic text       F   filename
B     bold text         S       text with non-breaking spaces
C     code text         X index entry
L     hyperlink         Z<>           null
E   character escape  B<<>>   More than one delimiter is ok

COMMON MAIN SECTIONS               
NAME        BUGS/CAVEATS
SYNOPSIS    DIAGNOSTICS
DESCRIPTION DEPENDENCIES
COPYRIGHT   LICENSE
SEE ALSO    AUTHOR

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Saturday 20th May, 2006 01:31:33 AEST

Perl Programming with Testing

Until I've come up with something to add, for the moment the best way I can think to describe testing with perl is to reference several excelent sources of documentation on the subject:

Test::Harness - Run Perl standard test scripts with statistics
Test::Simple - Basic utilities for writing tests
Test::More - yet another framework for writing test scripts
prove - A command-line tool for running tests against Test::Harness

Those provide an excellent basis for constructing and running tests in perl.

Once the processes in those documents and modules have become familiar there are a number of excellent tools that can be used to make testing even more powerful:

Devel::Cover - Code coverage metrics for Perl
perlcritic - Command-line interface to critique Perl source
Test::WWW::Mechanize - Testing-specific WWW::Mechanize subclass

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Saturday 20th May, 2008 00:53:53 AEST

Starting a Catalyst application

Overview

Early experiences with using the Catalyst Web Framework tend to suggest that there are a few common starting points when deploying a new Catalyst application.

These steps take the 'complete' application harness (which is essentially a completely working but completely empty web application) and add a few features to get a little content and functionality into the application.

To begin with it's worth noting that most of documentatin for Catalyst is in the POD - so you can do a lot worse then point your web browser at Catalyst when you're looking for help. The next two resources I've found most useful are the mailing list and the Developer Community Site.

Another invaluable starting point is the Catalyst Tutorial.

Once an application has been created it's time to start adding functionality, so without further ado:

catalyst.pl TestApplication

Using Template Toolkit for the primary View

Often the Template Toolkit will be all you need to manage the View output of an application.

The easiest way to start using Template Toolkit as the default view is to create a view using the Template Toolkit View helper:

./script/testapplication_create.pl view TT TT

Once this is done, the view is most easily accessed by using the DefaultEnd Plugin which will direct all responses without content in the body to the first View. This is done by adding DefaultEnd to the use Catalyst part of lib/TestApplication.pm

use Catalyst qw/ -Debug
                 ConfigLoader
                 Static::Simple
                 DefaultEnd
               /;

A few notes on use of the Template Toolkit view:

A more complex template skeleton site can be automatically generated using the Catalyst::Helper::View::TTSite plugin: ./script/testapplication_create.pl view TT TTSite

If another View is added to the application all is not lost - the DefaultEnd plugin has a view configuration directive:

# In the YAML configuration file: testapplication.yml
---
name: TestApplication
view: TT

# OR

# In the application module: lib/TestApplication.pm
__PACKAGE__->config( name => 'TestApplication',
                     view => 'TT',
                   );

Changing the default page to be a 401: Not Found

Despite ''looking nice'' returning a '''200 Ok''' for any page requested from a website is very poor form causing all sorts of problems - for instance when spiders start walking through your site. As the default behaviour of the Catalyst base application is to do just that, this should be fixed: in lib/TestApplication/Controller/Root.pm:

#
# Output a friendly welcome message
#
sub default : Private {
    my ( $self, $c ) = @_;

    # If we reach here, the correct reponse is a 404
    $c->response->status(404);

    # Hello World
    $c->response->body( $c->welcome_message );
}

Of course removing the default Catalyst welcome page is also a good idea but can be done later when you get around to putting content into the site.

Adding a wrapper for the View

Because I didn't use the TTSite helper in this example I'll manually add a Template Toolkit wrapper to make building the rest of the pages a little easier. First - add some configuration to the main application (lib/TestApplicatoin.pm):

__PACKAGE__->config( name => 'TestApplication',
                     view => 'TT',
                     'View::TT' => {
                        INCLUDE_PATH => [
                          __PACKAGE__->path_to('templates'),
                          ],
                        WRAPPER => 'site/wrapper.tt',
                        }
                   );

This sets up a template include path in a templates directory in the root application directory. In addition a wrapper is defined. A Template Toolkit wrapper is a template which is wrapped around all content rendered by the library. When writing a wrapper the content of the page will be inserted where the [% content %] directive is placed. An example wrapper is as follows:

[% DEFAULT title = c.config.name %]


  [% title %]


[% title %]
[% content %]

Adding a root index page

Having started the server responding with 404 Not Found perhaps it would be nice to see at least one page that wasn't an error page. To add an index/root/welcome page add a private method called ''index'' to the Root controller:

sub index : Private {
    my ( $self, $c ) = @_;

    # Simple index
    $c->{stash}->{title} = "Index";
    $c->{stash}->{text} = qq{Welcome to the application - nothing to see here yet};
    $c->{stash}->{template} = "text.tt";
}

In this case the ''text.tt'' template is very simple:

[% text %]

For further information on the automagically called private methods of controllers see Catalyst::Manual::Intro.

Recap...

By this point we have a base catalyst application with the following additions: - Template Toolkit is being used to manage rendering for Views - Changed the default behaviour for the application to return 404 Not Found instead of 200 Ok - Added a wrapper to the Template Toolkit to facilitate templated content - Added an index page This is an excellent starting point to implementing actual functionality with a minimum of effort so could well be a good place to stop this article, but because the MVC framework is all about web applications with backend databases it would be remiss of me not to include a reference to a data source (or Model, as data sources are called in the MVC world).

Adding a SQLite Model

The available helper libraries for Catalyst include several ways to easily incorporate SQL databases - including Class::DBI and DBIx::Class. In this example I will use Class::DBI. In order to properly use data modelling it should not be a requirement that accessing the model is done through the catalyst application. By defining a Class::DBI library independant of the catalyst application and then referring to that library this can be done. If no library is defined for the database yet you may be able to use the automatic database interrogation done for you by Class::Loader. I'm using [http://www.sqlite.org/ SQLite] because it's very simple to do so - if you haven't seen SQLite before go have a look. :) To start with - create an SQLite database (in this case in a {{db}} subdirectory):

BEGIN TRANSACTION;
CREATE TABLE foo (
    bar VARCHAR(132)
);
INSERT INTO "foo" VALUES('sdfasd');
INSERT INTO "foo" VALUES('sdfasd');
INSERT INTO "foo" VALUES('sdfasd');
INSERT INTO "foo" VALUES('sdfasd');
INSERT INTO "foo" VALUES('sdfasd');
COMMIT;

Then add the model:

$ ./script/testapplication_create.pl model AppDB CDBI dbi:SQLite:/path/to/TestApplication/db/db.sqlite

Once this is done automagically it's helpful for portability to kill that hardwired path to the database - modify the dsn in the created AppDB.pm file from:

    dsn           => 'dbi:SQLite:/path/to/db.sqlite',

to:

    dsn           => 'dbi:SQLite:' . 
                     TestApplication->path_to('db') .
                     '/db.sqlite',

And that's it... No really... To access the Model inside the application use the model method:

my $foo_model = $c->model('AppDB::Foo');
my @foo_rows = $foo_model->retrieve_all();

And all is well...

And there we have it - a dully functioning web application (with no pages or functoins to speak of) but incorporating a fully functioning SQL database abstracted through a data model and using a powerful templating language to minimise data rendering complexity. On top of that you have a full MVC application ''out-of-the-box''. At this point it's probably time to start adding some useful functionality to the application, have fun with that.

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Friday 19th May, 2006 21:46:19 AEST

Which perl module files are being used?

And now for a super-short article with a fast answer to a problem - on the other hand, it's a problem I really needed a quick answer to the other day that I couldn't find. So here's the article:

Question: Which perl module files are being used during the running of a script?

This can be very useful because it's sometimes very helpful to know which of the installed versions of a module are being used - in my case I needed to build a set of libraries to deploy onto a server on which I could not easily build libraries. The problem was finding out when my script was quietly grabbing a module from the core perl installation instead of my library bundle.

A couple of interesting discussions on PerlMonks on the matter:

There were a couple of different approaches discussed, the most complex of which were ''re-following'' the @INC array to try and find which library ''would'' be used. The problem with that approach is that it's a guess about what will be used rather than a report of what was used.

It turns out there's a very very simple way...

Answer

One of the Perl predefined variables is %INC (not to be confused with the library search path @INC).

As per the perldoc:

%INC The hash %INC contains entries for each filename included via the "do", "require", or "use" operators. The key is the filename you specified (with module names con- verted to pathnames), and the value is the location of the file found. The "require" operator uses this hash to determine whether a particular file has already been included. If the file was loaded via a hook (e.g. a subroutine reference, see "require" in perlfunc for a description of these hooks), this hook is by default inserted into %INC in place of a filename. Note, however, that the hook may have set the %INC entry by itself to provide some more specific info.

So to get a run-time report of what modules are in use, and where the source files were, just print out %INC:

1 use Data::Dumper; 2 print Data::Dumper:;Dumper(\%INC);

For example:

1 $ perl -MData::Dumper -MEnglish -MCGI -e 'print Data::Dumper::Dumper(\%INC)' 2 $VAR1 = { 3 'warnings/register.pm' => '/usr/lib/perl5/5.8.6/warnings/register.pm', 4 'bytes.pm' => '/usr/lib/perl5/5.8.6/bytes.pm', 5 'Carp.pm' => '/usr/lib/perl5/5.8.6/Carp.pm', 6 'XSLoader.pm' => '/usr/lib/perl5/5.8.6/i386-linux-thread-multi/XSLoader.pm', 7 'English.pm' => '/usr/lib/perl5/5.8.6/English.pm', 8 'Exporter/Heavy.pm' => '/usr/lib/perl5/5.8.6/Exporter/Heavy.pm', 9 'vars.pm' => '/usr/lib/perl5/5.8.6/vars.pm', 10 'strict.pm' => '/usr/lib/perl5/5.8.6/strict.pm', 11 'Exporter.pm' => '/usr/lib/perl5/5.8.6/Exporter.pm', 12 'constant.pm' => '/usr/lib/perl5/5.8.6/constant.pm', 13 'warnings.pm' => '/usr/lib/perl5/5.8.6/warnings.pm', 14 'CGI/Util.pm' => '/usr/lib/perl5/5.8.6/CGI/Util.pm', 15 'overload.pm' => '/usr/lib/perl5/5.8.6/overload.pm', 16 'CGI.pm' => '/usr/lib/perl5/5.8.6/CGI.pm', 17 'Data/Dumper.pm' => '/usr/lib/perl5/5.8.6/i386-linux-thread-multi/Data/Dumper.pm' 18 };

Here's the perldoc on PerlDoc:perlvar.

A note on a gotcha

Of course, there must be a gotch...

Modifying %INC is a common way to trick Perl into believing that a module has already been loaded (for instance when using something like Test::MockObject) but when that happens the hash value set is usually not a path to a file. That said - this method is not going to be all that reliable if used in an environment in which munging of %INC is going on.

Posted by Bradley Dean | Permalink | Categories: Perl, Programming