The idea: a script which I can call with partial code or currency names and an amount which will then try and tell me what the current converted value is. If there's something other than a single matching currency let me know that nothing or too many things matched.

Here's the script (also attached):

 1  #!/usr/bin/env perl
 2
 3  use strict;
 4  use warnings;
 5
 6  use Finance::Quote;
 7  my $q = Finance::Quote->new();
 8
 9  my %trans = ();
10  @trans{qw(from to)} = @ARGV;
11
12  # Copied this patter from Finance::Quote::currency to be consistent
13  $trans{from} =~ s/^\s*(\d*\.?\d*)\s*//;
14  my $amount = $1 || 1;
15
16  # Search out currencies
17  for my $key (qw( from to )) {
18    # Is it a code or a name?
19    my $currencies;
20    if ( $trans{$key} =~ /^[A-Z]{2,}$/ ) {
21      # Guess code
22      $currencies = $q->currency_lookup( code => qr/$trans{$key}/ );
23    }
24    else {
25      # Guess name
26      $currencies = $q->currency_lookup( name => qr/$trans{$key}/i );
27    }
28    if ( scalar keys %{$currencies} == 1 ) {
29      my $real_key = "real_${key}";
30      ($trans{$real_key}) = keys %{$currencies};
31      printf "%s : %s (%s)\n", $key
32                             , $trans{$real_key}
33                             , $currencies->{$trans{$real_key}}->{name};
34    }
35    elsif ( scalar keys %{$currencies} > 1 ) {
36      print "Multiple currency matches for ${key}:\n";
37      print " * $_ (" . $currencies->{$_}->{name} . ")\n" for keys %{$currencies};
38    }
39    else {
40      print "No currency matches for ${key}: " . $trans{$key} . "!\n";
41    }
42  }
43
44  # If real from and to exist run the conversion
45  if ( exists $trans{real_from} && exists $trans{real_to} ) {
46    print "${amount} $trans{real_from} => "
47        . $q->currency("${amount}$trans{real_from}", $trans{real_to})
48        . " $trans{real_to}\n";
49  }

And here's some examples of this script in action:

# Some Australian dollars to pounds
# (name searches only)
$ ./conv.pl 42aus brit
from : AUD (Australian Dollar)
to : GBP (British Pound)
42 AUD => 18.8454 GBP

# And the some pounds to US dollars
# (a name and a code search, because US is all-caps)
$ ./conv.pl brit US
from : GBP (British Pound)
to : USD (U.S. Dollar)
1 GBP => 1.4225 USD

# Oops - a typo on pounds
$ ./conv.pl GPB euro
No currency matches for from: GPB!
to : EUR (Euro)

# And now, to make the British cry...
$ ./conv.pl GBP euro
from : GBP (British Pound)
to : EUR (Euro)
1 GBP => 1.1317 EUR

# And finally, too many matches:
$ ./conv.pl egypt ven
from : EGP (Egyptian Pound)
Multiple currency matches for to:
 * SIT (Slovenian Tolar)
 * VEB (Venezuelan Bolivar)

Finance::Quote uses the Yahoo Currency Converter website so the new function needed to report a compatible list of currencies.

Having developed the search method against a statically stored list of currencies (as I hadn't implemented the live extraction yet) I then discussed options with the other developers and decided that a hard coded list was a better approach given that the list wasn't going to change very often.

Here's a basic idea of usage:

1    $currencies_by_name = $q->currency_lookup( name => 'Australian' );
2    $currencies_by_code = $q->currency_lookup( code => qr/^b/i      );
3    $currencies_by_both = $q->currency_lookup( name => qr/pound/i
4                                             , code => 'GB'         );

The return value is a hash of hash-refs - for example for the first query above:

1    $VAR1 = { AUD => { name => "Australian Dollar" } }

I had added a module to contain the currency list (Finance::Quote::Currencies). Once the search code was complete I then added a test to compare the static list with a live list and implemented Finance::Quote::Currencies::fetch_live_currencies(). This function uses HTML::Parser to extract the currency list. If the currency list changes this function can be used to easily updated the stored list.

These changes are currently available on the currency_lookup branch on github. Coming to a CPAN release in your town real soon now.

Answer: It should be simple, but there's no such formatter in MoinMoin (and no way I know to just say 'embed this text as-is') but surely this must be one of the simplest MoinMoin Parsers that could be written - so let's have a look...

The CSV Parser looked quite simple - from there it became clear that all you really need to do is initialise a couple of attributes in the constructor and then write the simplest-case format method:

 1  class Parser:
 2      """ Format HTML as HTML
 3      """
 4
 5      def __init__(self, raw, request, **kw):
 6          """ Store the source text.
 7          """
 8          self.raw = raw
 9          self.request = request
10          self._ = request.getText
11
12      def format(self, formatter):
13          """ Send it back as it came
14          """
15          self.request.write(self.raw)

Load this into your wiki plugin directory as data/plugin/parser/html.py directory, start a document with #FORMAT html and it just works. This probably wouldn't be a great feature for a public wiki, but it's nice to be able to roll your own so easily. :)

1    # ... determine second, minute, hour, day, month, year
2    my $epoch_seconds = strftime('%s', $second, $minute, $hour
3                                , $day, $month - 1, $year - 1900 );

Which of course works fine under *nix - but it turns out (a bit obviously with hindsight) that the windows strftime does not support '%s'. Unfortunately this does not result in any sort of error - rather you just end up with the literal string '%s'

Time::Local to the rescue - this module implements the reverse of the builtin localtime and gmtime functions.

Here's a version of the above code which works cross-platform:

1    use Time::Local;
2    # ... determine second, minute, hour, day, month, year
3    my $epoch_seconds = timelocal( $second, $minute, $hour
4                                 , $day, $month - 1, $year);

As an aside - the module extends the interpretation of the year to be more user friendly (so you can use fully qualified years, the familiar '$year - 1900' or two-digit years with a rolling window). It's probably better to stick with the fully qualified year - particularly when using old dates as it's possible that '$year - 1900' will end up in a sliding window and drop you in the wrong century.

(For other approaches read `perldoc -q epoch`)

Comment by: Bradley Dean at Thu, 22 Jan 2009 14:18:27 (GMT)

So why isn't this problem more apparent? When I first started looking around for the causes I couldn't really see much. One reason is probably that some developers 'just know' that their strftime has '%s' (not to mention that their time is based on the epoch) while others have never run into the epoch so would never call on '%s'.

The perl strftime documentation refers the reader to the local strftime man page for conversion specifiers, while others (I've looked at python, ruby and php) give a list of available conversion specifiers. I've checked time.strftime in python and it support '%s' (but it's undocumented) which I suspect will be true for most other languages as they'll be wrapping the system strftime rather than reimplimenting it.

I've written a patch for the top in procps which adds j/J command line and run-time options which essentially mirror the u/U filters.

I've sent this in to the procps maintainers just in case they want to include it in the trunk.

The patch is here: procps-3.2.7-exclude-user-patch.

The full source bundle is here: procps-3.2.7-exclude-user.tgz.

A quick look revealed that I had ended up with some ISO-8859-1 (ie Latin-1) character encodings within filenames, which was all very well before I switched to UTF-8!.

Contemplating possible temporary resetting of my locale (but what about my newer files that include UTF-8 encodings?) or some sort of simple filter that dropped characters I chanced upon Unicode Tools and a neat list of tools available in various languages.

In this case perl's Encode came to the rescue with this rather short and nicely effective script:

 1  #!/usr/bin/env perl
 2
 3  #
 4  # For each filepath given on stdin, convert the filename from
 5  # iso-8859-1 to utf-8 (or try to!)
 6  #
 7
 8  use strict;
 9  use warnings;
10  use Encode;
11
12  while ( my $orig_filepath = <STDIN> ) {
13    if ( $orig_filepath =~ /[\xA0-\xFF]/ ) {
14      chomp $orig_filepath;
15      my $new_filepath = $orig_filepath;
16      Encode::from_to($new_filepath, 'iso-8859-1', 'utf-8')
17        or die $!;
18      print "$orig_filepath ---> $new_filepath\n";
19      rename $orig_filepath, $new_filepath;
20    }
21  }

The check for characters between 0xA0 and 0xFF is far from perfect for general use (indeed differenciating between the various ISO-8859-*'s would be a bit of a nightmare) but as I knew my encodings were limited to ISO-8859-1 I got away with a bit of a shortcut there.

Comment by: Bradley Dean at Sat, 1 Nov 2008 09:44:11 (BST)

Of course being somewhat late at night I missed the obvious blunder in the logic - there are characters in the range 0xA0-0xFF in UTF-8 encoded strings so rerunning can cause a bit of a mess.

I'm still thinking about this - the UTF-8 multibyte marker byte is in the range 0xC0 to 0xFD. These are all legal ISO-8859-1 character codes so I think I need to draw some sets out on a piece of paper and find the mutually exclusive bits.

Comment by: Bradley Dean at Sun, 2 Nov 2008 00:33:40 (BST)

Given that UTF-8 character bytes for non-ASCII characters fall within the range 0x80 to 0xFD, and given that this overlaps with the character bytes used by ISO-8859-1, there's not a way to differenciate a high-byte string just by looking at individual bytes.

In my scenario (text will either be plain ASCII, ISO-8859-1 or UTF-8) the best test seems to be to try and interpret the string as UTF-8. If errors occur it's going to be ISO-8859-1, if not it's going to be UTF-8.

The module is only useful if any exception can be silently caught and the code in question re-run, but in that case it makes it very easy to write the loop in question:

1    use Die::Retry qw( retry );
2    my $retval = retry( sub { some_func_with_exceptions($param1, $param2) }
3                            , times => 3
4                            , delay => 0
5                            );

In that example the code ref:

1  sub { some_func_with_exceptions($param1, $param2) }</pre>

is called up to three times if exceptions are thrown. There is no delay between retries. Finally - if a successful call is made to the code ref $retval will get the (scalar context) return value of the code ref call.

I'm submitting this to CPAN and have already set it up on GitHub and Ohloh.

Comment by: Bradley Dean at Mon, 20 Oct 2008 22:56:49 (BST)

I've just found Attempt which does exactly what I coded for Die::Retry - in fact both the code and the syntax are practically identical. So dies Die::Retry.

Albeit without the "blur" label! :)

Of course I'm back to configuring most things by hand again (wireless, mounting usb drives, X) - but that has always held a certain perverse enjoyment for me anyway!

Many years ago we were setting up a server monitor and it was decided that we'd come up with an executive summary style - that being a simple thumbs-up:

Or a thumbs-down:

Somewhere along the way the system developed a bit of an attitude and somehow the thumbs-up became a little bit more like....

Don't really know why I've posted that but a recent conversation lead to re-building those simple images and I thought I'd put them up in lights. :)