July 13, 2008

Using the newest zipped pytz on GAE

I posted an entry about how to use zipped pytz on GAE[1].

1. http://takashi-matsuo.blogspot.com/2008/07/using-zipped-pytz-on-gae.html

It works well with older versions of pytz. Stefano pointed out that my method doesn't work with the newest pytz distribution. This article describes how to use the newest zipped pytz on GAE.

First, retrieve the newest pytz from pypi[2] and extract the archive.

2. http://pypi.python.org/pypi/pytz/

$ tar xjf pytz-2008c.tar.bz2
$ cd pytz-2008c/pytz
$ zip -q zoneinfo.zip `find zoneinfo -type f ! -name '*.pyc' -print`
$ rm -rf zoneinfo

After that, you have to edit pytz/__init__.py and modify open_resource function to use zipped zoneinfo database like following:
(pkg_resources stuff were struck out. Thank you again Stefano!)

def open_resource(name):
"""Open a resource from the zoneinfo subdir for reading.

Uses the pkg_resources module if available.
import zipfile
from cStringIO import StringIO

if resource_stream is not None:
return resource_stream(__name__, 'zoneinfo/' + name)

name_parts = name.lstrip('/').split('/')
for part in name_parts:
if part == os.path.pardir or os.path.sep in part:
raise ValueError('Bad path segment: %r' % part)
zoneinfo = zipfile.ZipFile(os.path.join(os.path.dirname(__file__),
return StringIO(zoneinfo.read(os.path.join('zoneinfo', *name_parts)))

Then, the only thing to do is to copying your original pytz directory into your application directory.

$ cd ../
$ cp -r pytz /your/application/directory

If you'd like to avoid using CPU to unzip zoneinfo data every time, perhaps you could use following function:

from google.appengine.api import memcache
import logging
import pytz
from pytz import timezone

def getTimezone(tzname):
tz = memcache.get("tz:%s" % tzname)
tz = None
logging.debug("timezone get failed: %s" % tzname)
if tz is None:
tz = timezone(tzname)
memcache.add("tz:%s" % tzname, tz, 86400)
logging.debug("timezone memcache added: %s" % tzname)
logging.debug("timezone memcache hit: %s" % tzname)

return tz

Happy coding :-)

July 12, 2008

Using zipped pytz on GAE

To handle timezone of all over the world crrectly, I think it is the most easy way to use pytz[1]. Besides that, there are many modules and programs which depends on pytz modes, so I believe that quite a few people would like to use pytz on GAE. However, pytz contains over 500 files in its zoneinfo directory, so it often bears on the 1000 files limit.

1. http://pytz.sourceforge.net/

This is an instruction about how to use zipped pytz on GAE environment. In this article, we use my_zipimport.py[2] provided by Guido van Rossum.

2. http://code.google.com/p/googleappengine/issues/detail?id=161#c19

First, make your own pytz.zip as following:

$ cd pytz-2006p
$ zip -q pytz.zip `find pytz -type f ! -name '*.pyc' -print`

Secondly, make sure to put my_zipimport.py and pytz.zip into your GAE app directory.
Lastly, put following code into your script.

import my_zipimport
import sys

sys.path.insert(0, 'pytz.zip')

That's all. Now you can use pytz module seamlessly.

July 4, 2008

Hackathon kit

Google I/O のプレゼンを和訳したこれ使ってセミナーしても良い?本家グループ にメールしたら、ある US の googler からメールが来まして、なんと hackathon kit を提供してくれるようです。 わーい。

という亊で 7月の終わりか8月の始めに hackathon or campfire やりたいな...


Google-App-Engine-Japan グループ で Meet-at というオープンソースプロジェクトを始める際の共同開発者を募集したら結構反応がありました。かなり強力なメンバーが揃ったので、個人的にはとても期待しています。

早速 7/5 にキックオフミーティングをやる予定です。どの技術を使おうとか相談するのもなかなか楽しいですが早くコーディングが始まらないかな...