Fri Mar 28 00:22:03 GMT 2008

AmazonS3Store 1.1 - Handle truncated responses

Something I should probably have noticed before but Amazon S3 list_bucket responses can be truncated if theres a lot of entries in a bucket.

If truncation has occurred the is_truncated attribute is set to be true on the response.

Follow-on requests can be made by setting the marker option to the call to list_bucket to be the key of the last entry received:

1  options = { 'marker' : list_bucket_response.entries[-1].key }
2  list_bucket_response = conn.list_bucket(bucket, options)

Here's the release of AmazonS3Store 1.1 with the changes: AmazonS3Store-1.1.tar.gz.

Version 1.1 - Fri Mar 28 00:02:04 GMT 2008
 * Added internal function __list_bucket_iter(cmdopts, cfg, conn)
   which wraps the connection list_bucket. The function is an iterator and
   makes multiple requests where truncated responses are detected.

Posted by Bradley Dean | Permalink | Categories: Python, Programming

Thu Mar 27 00:59:12 GMT 2008

LWP::UserAgent fails with '500 Line too long (limit is 4096)'

This one caught somewhat by surprise - mostly because of the unusual use of a 500 (Internal Server Error) code by LWP::UserAgent to flag a client-side configuration problem.

The answer, as with so many things perl, lay with the PerlMonks in this article: Mysterious LWP status 500 Line too long.

To summarise: Net::HTTP has a default maximum header line length of 4096. This can be changed by passing a MaxLineLength option to the constructor - and happily LWP::UserAgent has an internal hash which can be tweaked to set the desired maximum line length:

1  my %OPTS = @LWP::Protocol::http::EXTRA_SOCK_OPTS;
2  $OPTS{MaxLineLength} = 8192; # Or however large is needed...
3  @LWP::Protocol::http::EXTRA_SOCK_OPTS = %OPTS;

Posted by Bradley Dean | Permalink | Categories: Perl, Programming

Sun Mar 16 11:33:17 GMT 2008

pyprove 1.3 - __import__ compatability between python versions 2.4 and 2.5

__init__ changed between versions 2.4 and 2.5 of python - in 2.5 keyword arguments are available.

I've just used the old-style call which is supported in 2.5:

1  -    imported_module = __import__(module_fqname, fromlist = [1])
2  +    imported_module = __import__(module_fqname, globals(), locals(), [1])
Version 1.3 - Sun Mar 16 11:24:14 GMT 2008
 * Changed calls to __import__ to be compatible between python versions
   2.4 and 2.5 (in 2.5 __import__ supports keyword arguments)


Posted by Bradley Dean | Permalink | Categories: Python, Programming

Fri Mar 14 22:04:01 GMT 2008

pyprove 1.2 - 'has no tests' bugfix

And now another release for a bugfix:

Version 1.2 - Fri Mar 14 21:57:01 GMT 2008
 * When doctest.DocTestSuite throws a ValueError (has no tests)
   recover from the exception and move on


Posted by Bradley Dean | Permalink | Categories: Python, Programming

Fri Mar 14 19:52:34 GMT 2008

pyprove 1.1 - better search path configuration

A full several minutes between the first and second release, not too bad...

As per the changelog:

Version 1.1 - Fri Mar 14 19:36:15 GMT 2008
 * Change appending of library_prefix_paths entries to
   inserting them at the beginning of sys.path to make sure
   local source is found before source installed other places
   in the path

 * Pre-pend os.getcwd() to sys.path to help finding libraries
   in the right place without having to set PYTHONPATH

Here it is: pyprove-1.1.tar.gz

Posted by Bradley Dean | Permalink | Categories: Python, Programming

Fri Mar 14 19:20:24 GMT 2008

pyprove - recursively run tests like perl prove

I know this exists in various guises around the place but I've been coming back to wanting something easy to use like perl prove when writing python.

A while back I was doing some work on Zope and they have which does just that - but it's somewhat specific to Zope.

So I've sat down and come up with pyprove - the idea being that it should be possible to run the script within a source code tree and have it go out and find unittest and doctest tests and run them all.

Thus far it's pretty simple - no command line options are available and it only runs from the current directory. It also requires that PYTHONPATH is set if tested modules are not in the current search path.

Here's a source distribution: pyprove-1.0.tar.gz.

(Debugging is greatly facilitated by re-including the commented out logging configuration line.)

Posted by Bradley Dean | Permalink | Categories: Python, Programming