tag:blogger.com,1999:blog-21444700517776365472024-03-19T12:23:37.083+09:00Moon blue diarytmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.comBlogger41125tag:blogger.com,1999:blog-2144470051777636547.post-20002253451851459452010-05-14T15:39:00.002+09:002010-05-14T17:04:29.998+09:00Now Kay Framework has GAETestBase bundled!What I've been working on recently is to incorporate <a href="http://takashi-matsuo.blogspot.com/2010/05/introducing-gaetestbase.html">GAETestBase</a> to <a href="http://code.google.com/p/kay-framework/">Kay Framework</a>.<br />
<br />
I've just finished the adapting work and bundled GAETestBase works very nicely with kay framework. You just need to change the base class of your testcase to "kay.ext.testutils.gae_test_base.GAETestBase" like:<br />
<br />
<pre class="prettyprint">from kay.ext.testutils.gae_test_base import GAETestBase
class DummyTest1(GAETestBase):
#...
def test_foo(self):
# ...
# ...
</pre><br />
By default, when you run your tests via gaeunit, the web testrunner will collect testcases from all of your applications defined in settings.INSTALLED_APPS variable. For now, you cannot change this behavior, but in the near future, I'll add a feature for configuring which test to run via the gaeunit web testrunner.<br />
<br />
The recent version of Kay has a url mapping for web testrunner, but your app.yaml may lack of it. In such a case, please add following entry to your app.yaml:<br />
<br />
<pre class="prettyprint">- url: /_ah/test.*
script: kay/ext/testutils/gaeunit.py
login: admin</pre><br />
As you can see, you can run your test by visiting a URL like "/_ah/test".<br />
<br />
As a note, all of your tests will be called in parallel, so it might leads to unexpected test failures. So you need to write your test functions independent from other tests.<br />
<br />
Please see <a href="http://takashi-matsuo.blogspot.com/2010/05/introducing-gaetestbase.html">my previous post</a> for details about how to configure GAETestBase.<br />
<br />
Happy testing!tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com1tag:blogger.com,1999:blog-2144470051777636547.post-72861717590670730872010-05-14T14:30:00.002+09:002010-05-14T14:45:42.789+09:00How to download souce code of your application written with kayLet me introduce a small trivia to you. What if your harddisk crashes and you lost your source code, hopefully, you might use some source code management system such as cvs, subversion, mercurial or git, in such a case, you can retrieve your source code back. But what if the crash occurs before your first commit, or your repository also crashes?<br />
<br />
Don't give up!<br />
<br />
Recent version of Kay Framework has remote_api and deferred handler by default, so there is still a chance for retrieving it back to you.<br />
<br />
This idea is originally come from <a href="http://stackoverflow.com/questions/2479087/can-i-restore-my-source-code-that-has-been-uploaded-into-google-appengine/">this thread</a> in stackoverflow.<br />
<br />
<ul><li>First, create a brand new project with kay's manage.py script and edit your app.yaml. Just put your application-id on the top of the file.<br />
<br />
<pre class="prettyprint">$ python ~/work/kay/manage.py startproject downloadcode
Running on Kay-0.10.0
Finished creating new project: downloadcode.
$ cd downloadcode
$ vi settings.py
</pre></li>
<li>Second, create a file named "restore_code.py" as follows:<br />
<br />
<pre class="prettyprint">import os
from google.appengine.ext import db
expr = """
[type(
'CodeFile',
(__import__('google.appengine.ext.db').appengine.ext.db.Expando,),
{})(
name=dp+'/'+fn,
data=__import__('google.appengine.ext.db').appengine.ext.db.Text(
open(dp + '/' + fn).read()
)
).put() if not dp.startswith('../kay') else None
for dp, dns, fns in __import__('os').walk('..')
for fn in fns]
"""
class CodeFile(db.Model):
name = db.StringProperty(required=True)
data = db.TextProperty(required=True)
@classmethod
def kind(cls):
return cls.__name__
def restore():
for cf in CodeFile.all():
fpath = cf.name.replace("../", "./_restored_files/")
dirname = os.path.dirname(fpath)
if not os.path.exists(dirname):
os.makedirs(dirname)
fh = open(fpath, "w")
fh.write(cf.data)
fh.close()
def scan():
from google.appengine.ext.deferred import defer
defer(eval, expr)
</pre></li>
<li>Next, connect production server with "python manage.py rshell command."<br />
<pre class="prettyprint">$ python manage.py rshell
Running on Kay-0.10.0
Username:matsuo.takashi
Password:
Interactive Kay Shell with RemoteDatastore.
-----------------WARNING--------------------
Please be careful in this console session.
-----------------WARNING--------------------
In [1]: import restore_code
In [2]: restore_code.scan()</pre>restore_code.scan() will scan filesystems on the production and create CodeFile entities in the datastore. This work will be done by deferred handler, so you may need to wait a bit, but the completion of this process won't be notified. So why don't you take a break here and have a cup of coffee.</li>
<li>Lastly, call "restore_code.restore()" function.<br />
<pre class="prettyprint">In [3]: restore_code.restore()
In [4]:</pre></li>
</ul><br />
Then, you can have your entire source code except for static files and configuration files(such as app.yaml, index.yaml, etc..) in a directory named "_restored_files".<br />
<br />
Yey, now you've got your code back!tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-15306764745102240342010-05-13T17:10:00.005+09:002010-05-14T17:08:57.121+09:00Introducing GAETestBaseA Japanese highly skilled engineer <a href="http://d.hatena.ne.jp/tagomoris/">tago-san</a> wrote an excellent module named GAETestBase. I'd like to introduce GAETestBase to you all. This entry is basically an English translation of <a href="http://d.hatena.ne.jp/tagomoris/20100512/1273671773">this article of his</a>.<br />
<br />
GAETestBase is a useful subclass of the standard unittest.TestCase. You can just write your test classes as usual by just extending GAETestBase class for utilizing this modules functionality.<br />
<br />
GAETestBase can be downloaded at <a href="http://code.google.com/p/appengine-py-testhelper/source/browse/trunk/gae_test_base.py">Google Code Repository</a>.<br />
<br />
<h1>What can I do with this module?</h1><div><ul><li>Run your tests via CLI</li>
<ul><li>You can run your tests without setting up necessary environments (e.g. stubs, env, etc..)</li>
<li>Tests will be use remote_api connection for accessing production datastore if you configure to do so in your TestCase.</li>
</ul><li>Run your tests via GAEUnit</li>
<ul><li>GAEUnit is a web-based test runner hosted at: <a href="http://code.google.com/p/gaeunit/">GAEUnit Google Code</a></li>
<li>You can run your test with web-browser in your development environment.</li>
<li><b>You can also run your tests with web-browser in your production environment. The test will be invoked under real production services(not memory based DatastoreFileStub) if you configure to do so.</b></li>
</ul><li>Run your tests with special overridden kind() method.</li>
<ul><li>You can use this overridden kind method just in your tests without any changes in your code.</li>
<ul><li>For example, entities of a model class "MyModel" will be stored as kind "t_MyModel".</li>
<li>Be careful with this special kind method in your TestCase. Please see an example bellow.<br />
<br />
<pre class="prettyprint">Key.from_path('MyModel', id) # NG!
Key.from_path(MyModel.kind(), id) # OK!</pre></li>
</ul><li>Especially useful with tests via remote_api and tests on production(GAEUnit) because you can run your tests without any data pollution.</li>
<li>Of course, you can suppress this behavior by configuration</li>
</ul><li>Clean up all the kinds that are used in your tests.</li>
<ul><li>You can delete all the kinds used in your test after running your test.</li>
<ul><li>Only the kinds which is accessed in a particular TestCase, will be deleted.</li>
<li>*accessed* here includes just reading. So if you read a existing kind, the kind will become a target of deletion.</li>
</ul><li><b><span class="Apple-style-span" style="color: red;">CAUTION:</span> This feature could be very dangerous if you disable overridden kind </b><b>method.</b></li>
</ul></ul><ul><ul><ul></ul></ul></ul></div><h1>Let's do it</h1><h3>Download necessary files and deployment</h3>You need to do following three things before start writing your tests.<br />
<ol><li>Deploy gaeunit.py</li>
<ul><li>Download the file and put it into your project directory.</li>
<li>You need to re-write this file for changing _LOCAL_TEST_DIR if your tests are not placed in test/*.py.</li>
</ul><li>Add two entries to your app.yaml</li>
<ul><li>Configure a gaeunit handler and remote_api handler as follows:<br />
<br />
<br />
<pre class="prettyprint">- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
- url: /test.*
login: admin # This is important if you deploy the test directory in production!
script: gaeunit.py</pre><br />
</li>
</ul><li>Create "test" directory and put gae_test_base.py into it, and edit constant variables in the file.</li>
<ul><li>You need to configure following variables:</li>
<ul><li>GAE_HOME: a path for google appengine SDK</li>
<li>PROJECT_HOME: a path for your target project</li>
<li>APP_ID: your application id</li>
<li>REMOTE_API_ENTRY_POINT: a path section of the URI of remote_api (e.g. "/remote_api" for above example).</li>
</ul></ul></ol><h3>Run your test in local env.</h3>Firstly, let's try running your test for local environment. This is a pretty ordinary test. "test_defs" here is just for model definition, so replace it with actual module with yours.<br />
<br />
<pre class="prettyprint">from gae_test_base import *
from test_defs import *
from google.appengine.ext import db
class DummyTest1(GAETestBase):
def test_put(self):
x1 = XEntity(x="x1")
k1 = x1.put()
self.assertEqual(db.get(k1).x, "x1")
def test_tx(self):
def tx1(x2key, x2, x3):
x2 = XEntity(key=x2key, x=x2)
x3 = XEntity(x=x3, parent=x2)
x2.put()
x3.put()
return (x2.key(), x3.key())
x2k = db.allocate_ids(db.Key.from_path(XEntity.kind(), -1), 1)[0]
k2, k3 = db.run_in_transaction(tx1, db.Key.from_path(XEntity.kind(), int(x2k)), "x2", "x3")
self.assertEqual(db.get(k2).x, "x2")
self.assertEqual(db.get(k3).x, "x3")</pre><br />
Actual tests are very simple as follows:<br />
<ol><li>test_put</li>
<ul><li>Create a new entity and put() it.</li>
<li>Getting an entity with returned key again, and compare values of two entities.</li>
</ul><li>test_tx</li>
<ul><li>Define a following function "tx1" for transaction</li>
<ul><li>receives a key for a root entity, a value for a property of the root entity, and a value for a property of the child entity as its arguments.</li>
<li>creates a new root entity and a child entity.</li>
<li>puts'em together</li>
<li>returns those two keys</li>
</ul><li>Get an id for a new root entity by invoking allocate_ids</li>
<li>Run tx1 under a transaction</li>
<li>Get two entities with the key returned from transaction call, and compare property values with expected one.</li>
</ul></ol>Here is an example output when running this testcase: <br />
<br />
<pre class="prettyprint">$ python2.5 dummy_test.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.020s
OK
$</pre><br />
Here is an example screenshot when running this testcase via GAEUnit:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://img.f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512213431.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="351" src="http://img.f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512213431.png" width="640" /></a></div><br />
<h3>Run your test via remote_api or on production</h3>Next, let's add a new testcase that uses the production datastore as its backend. Here is an example configuration for using remote_api via CLI, and using production datastore when deployed on appspot.<br />
<br />
<pre class="prettyprint">class DummyTest2(GAETestBase):
USE_PRODUCTION_STUBS = True
USE_REMOTE_STUBS = True
def test_put(self):
y1 = YEntity(y="y1")
k1 = y1.put()
self.assertEqual(db.get(k1).y, "y1")
def test_tx(self):
def tx1(y2key, y2, y3):
y2 = YEntity(key=y2key, y=y2)
y3 = YEntity(y=y3, parent=y2)
y2.put()
y3.put()
return (y2.key(), y3.key())
y2k = db.allocate_ids(db.Key.from_path(YEntity.kind(), -1), 1)[0]
k2, k3 = db.run_in_transaction(tx1, db.Key.from_path(YEntity.kind(), int(y2k)), "y2", "y3")
self.assertEqual(db.get(k2).y, "y2")
self.assertEqual(db.get(k3).y, "y3")</pre><br />
You can configure all the behavior by defining class attributes on your extended TestCase classes. The default value of all the configurable attributes is False when omitted. This new testcase is almost the same as the previous testcase except for a target model class.<br />
<br />
Here is an example output via CLI:<br />
<br />
<pre class="prettyprint">$ python2.5 dummy_test.py
....
----------------------------------------------------------------------
Ran 4 tests in 6.578s
OK
$</pre><br />
Now you can recognize that, this time, two testcases are called. You may asked your username and password on test that uses remote_api, though its omitted because there is a valid appcfg_cookies.<br />
<br />
Here is an output via GAEUnit:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512213433.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="351" src="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512213433.png" width="640" /></a></div><br />
You can also run these tests on appspot by just visiting http://app-id.appspot.com/test. After running these tests on appspot, you can see that entities which are used in your test are stored in the production datastore. A kind of these entities is "t_YEntity" (specially prefixed for test) like below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512212829.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="250" src="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512212829.png" width="640" /></a></div><br />
<h3>Cleaning up entities in your tests</h3>If you don't like entities created in your tests remains, you can write a testcase which deletes entities which are accessed in your tests like:<br />
<br />
<pre class="prettyprint">class DummyTest3(GAETestBase):
USE_PRODUCTION_STUBS = True
USE_REMOTE_STUBS = True
CLEANUP_USED_KIND = True
def test_put(self):
z1 = ZEntity(z="z1")
k1 = z1.put()
self.assertEqual(db.get(k1).z, "z1")
def test_tx(self):
def tx1(z2key, z2, z3):
z2 = ZEntity(key=z2key, z=z2)
z3 = ZEntity(z=z3, parent=z2)
z2.put()
z3.put()
return (z2.key(), z3.key())
z2k = db.allocate_ids(db.Key.from_path(ZEntity.kind(), -1), 1)[0]
k2, k3 = db.run_in_transaction(tx1, db.Key.from_path(ZEntity.kind(), int(z2k)), "z2", "z3")
self.assertEqual(db.get(k2).z, "z2")
self.assertEqual(db.get(k3).z, "z3")</pre><br />
You can run these test via CLI, <a href="http://f.hatena.ne.jp/tagomoris/20100512213434">via GAEUnit(dev)</a>, and <a href="http://f.hatena.ne.jp/tagomoris/20100512213435">via GAEUnit(prod)</a>.<br />
<br />
After running your tests, you can see there is no "t_ZEntity" on the datastore at all:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512212831.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="275" src="http://f.hatena.ne.jp/images/fotolife/t/tagomoris/20100512/20100512212831.png" width="640" /></a></div><br />
<h3>Per testcase cutomizing</h3>Here is an example for customizing kind prefix and clean up behavior.<br />
<br />
<br />
<pre class="prettyprint">class DummyTest4(GAETestBase):
USE_PRODUCTION_STUBS = True
USE_REMOTE_STUBS = True
CLEANUP_USED_KIND = False
KIND_PREFIX_IN_TEST = 'test'
def test_put(self):
z1 = ZEntity(z="z1")
k1 = z1.put()
self.assertEqual(db.get(k1).z, "z1")
def test_tx(self):
def tx1(z2key, z2, z3):
z2 = ZEntity(key=z2key, z=z2)
z3 = ZEntity(z=z3, parent=z2)
z2.put()
z3.put()
return (z2.key(), z3.key())
z2k = db.allocate_ids(db.Key.from_path(ZEntity.kind(), -1), 1)[0]
k2, k3 = db.run_in_transaction(tx1, db.Key.from_path(ZEntity.kind(), int(z2k)), "z2", "z3")
self.assertEqual(db.get(k2).z, "z2")
self.assertEqual(db.get(k3).z, "z3")</pre><br />
After running this test, you can see "test_ZEntity" is stored in the datastore(ruled out from deletion).<br />
<br />
<br />
<h3>Accessing the real data on production</h3><br />
Here is an example testcase for accessing the real data on prod.<br />
<br />
<br />
<pre class="prettyprint">class DummyTest5(GAETestBase):
USE_PRODUCTION_STUBS = True
USE_REMOTE_STUBS = True
CLEANUP_USED_KIND = False
KIND_NAME_UNSWAPPED = True
def test_put(self):
count = PEntity.all().count()
p1 = PEntity(i=count)
k1 = p1.put()
px = db.get(k1)
self.assertEqual(px.i, count)</pre><br />
If KIND_NAME_UNSWAPPED is set to True, you can suppress overriding kind method on db.Model. So, if you turn on USE_REMOTE_STUBS or USE_PRODUCTION_STUBS, your tests can access the real data on the production datastore.<br />
<br />
<h3>How to use this module with Kay Framework?</h3>Now Kay has GAETestBase bundled, so you can use it out of the box!<br />
For more details, please see <a href="http://takashi-matsuo.blogspot.com/2010/05/now-kay-framework-has-gaetestbase.html">this article</a> as well.<br />
<br />
<h3>Caution</h3>TODO: translate<br />
<br />
<br />
<h3>Far beyond</h3>TODO: translatetmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-16500782047814615302010-05-08T08:20:00.006+09:002010-05-20T10:39:02.933+09:00Creating an app for Google Apps Marketplace with Kay - part 1 (revised)<span style="font-size:200%">This entiry is obsoleted because Appengine SDK-1.3.4 was released, and it has a capability for OpenID authentication. I'll definitely start figuring an easy way for writing Google App Engine application for Google Apps Marketplace soon, so please wait a bit.</span><br />
<br />
<p>I've just implemented an authentication mechanism in Kay framework for Google Apps MarketPlace. So let me introduce how to deploy a marketplace application with Kay(repository version). <br />
<br />
<p style="font-color: #822">I have made a few more improvements in authentication system of Kay Framework, so I slightly change this article at 4:20PM on May 13th 2010JST.<br />
<h1>1. Create an app-id</h1><p>In this example, I've created a brand new application slot called 'marketplace-app'.<br />
<h1>2. Register a listing on the MarketPlace</h1><p>Firstly, you need to register yourself as a MarketPlace Vendor.<br />
http://www.google.com/enterprise/marketplace/<br />
<p>Click a link 'Become a vendor' in the bottom of the page above and you'll get an instruction.<br />
<p>Actually you need to click a link 'Sign in' at top-right corner of the page above, and fill the form.<br />
<p>Then, you can create your listing. Click a button 'Create a new listing' in your Vendor Profile page.<br />
<p>In this example, lets create an installable app, so check the first checkbox 'My product may be directly installed into Google Apps domains'.<br />
<p>You can fill other fields arbitrarily except for the field 'Manifest'.<br />
<p>Manifest xml here:<br />
<pre class="prettyprint"><?xml version="1.0" encoding="UTF-8" ?>
<ApplicationManifest xmlns="http://schemas.google.com/ApplicationManifest/2009">
<!-- Support info to show in the marketplace & control panel -->
<Support>
<!-- URL for application setup as an optional redirect during the install -->
<Link rel="setup" href="https://marketplace-app.appspot.com/a/${DOMAIN_NAME}/setup" />
<!-- URL for application configuration, accessed from the app settings page in the control panel -->
<Link rel="manage" href="https://marketplace-app.appspot.com/a/${DOMAIN_NAME}/admin" />
<!-- URL explaining how customers get support. -->
<Link rel="support" href="https://marketplace-app.appspot.com/a/${DOMAIN_NAME}/support" />
<!-- URL that is displayed to admins during the deletion process, to specify policies such as data retention, how to claim accounts, etc. -->
<Link rel="deletion-policy" href="https://marketplace-app.appspot.com/deletion-policy" />
</Support>
<!-- Name and description pulled from message bundles -->
<Name>Marketplace sample application</Name>
<Description>A simple application for the marketplace</Description>
<!-- Show this link in Google's universal navigation for all users -->
<Extension id="navLink" type="link">
<Name>Sample</Name>
<Url>https://marketplace-app.appspot.com/a/${DOMAIN_NAME}/start</Url>
<!-- This app also uses the Calendar API -->
<Scope ref="calendarFeed"/>
</Extension>
<!-- Declare our OpenID realm so our app is white listed -->
<Extension id="realm" type="openIdRealm">
<Url>https://marketplace-app.appspot.com/</Url>
</Extension>
<!-- Need access to the Calendar feed -->
<Scope id="calendarFeed">
<Url>https://www.google.com/calendar/feeds/</Url>
<Reason>This application shows the next Calendar event.</Reason>
</Scope>
</ApplicationManifest></pre><p>In this example, there is a calendar feed setting for 2 legged OAuth access scope. For more details with a Manifest file format, please see:<br />
http://code.google.com/googleapps/marketplace/manifest.html<br />
<p>After adding your listings, you will see a preview page for your app. There is an attractive 'Add it now' button on the right, but of course, you need to implement your application before adding this app.<br />
<p>Instead, click a link 'My Vendor Profile' on the top-right corner and you can see your listings with a link 'View OAuth Consumer Key'.<br />
<h1>3. Implementation</h1><p>Let's create a new project with Kay management script:<br />
<pre>$ python /some/where/kay/manage.py startproject marketplace-app
Running on Kay-0.10.0
Finished creating new project: marketplace-app.
$ cd marketplace-app
$ python manage.py startapp core</pre><p>settings.py:<br />
<pre class="prettyprint">INSTALLED_APPS = (
'core',
'kay.ext.gaema',
)
APP_MOUNT_POINTS = {
'core': '/',
}
MIDDLEWARE_CLASSES = (
'kay.sessions.middleware.SessionMiddleware',
'kay.auth.middleware.AuthenticationMiddleware',
)
AUTH_USER_BACKEND = "kay.auth.backends.gaema.GAEMABackend"
GAEMA_USER_MODEL = "core.models.User"
GAEMA_SECRETS = {
'google_consumer_key': 'your consumer key here',
'google_consumer_secret': 'your consumer key secret here',
}
IS_MARKETPLACE_APP = True
</pre><p>Replace GAEMA_SECRETS with actual values. You can see consumer key and consumer key secret with clicking the link 'View OAuth Consumer Key' mentioned above. The last attribute 'IS_MARKETPLACE_APP' is important for this authentication system work normaly.<br />
<p>core/models.py:<br />
<pre class="prettyprint"># -*- coding: utf-8 -*-
# core.models
from google.appengine.ext import db
from kay.ext.gaema.models import GAEMAUser
# Create your models here.
class User(GAEMAUser):
pass
</pre><p>Information of users will be stored into this model.<br />
<p>core/urls.py:<br />
<pre class="prettyprint"># -*- coding: utf-8 -*-
# core.urls
#
from kay.routing import (
ViewGroup, Rule
)
view_groups = [
ViewGroup(
Rule('/a/<domain_name>/setup', endpoint='domain_setup',
view='core.views.domain_setup'),
Rule('/a/<domain_name>/admin', endpoint='domain_admin',
view='core.views.domain_admin'),
Rule('/a/<domain_name>/support', endpoint='domain_support',
view='core.views.domain_support'),
Rule('/a/<domain_name>/start', endpoint='domain_start',
view='core.views.domain_start'),
Rule('/deletion_policy', endpoint='deletion_policy',
view='core.views.deletion_policy'),
)
]
</pre><p>core/views.py:<br />
<pre class="prettyprint"># -*- coding: utf-8 -*-
"""
core.views
"""
from werkzeug import (
unescape, redirect, Response,
)
from kay.utils import render_to_response
from kay.ext.gaema.utils import get_gaema_user
from kay.auth.decorators import login_required
def index(request):
return render_to_response('core/index.html', {'message': 'Hello'})
def domain_setup(request, domain_name):
callback = request.args.get('callback')
if callback is None:
return Response("No callback supplied.")
return redirect(callback)
def domain_admin(request, domain_name):
return Response("%s administration." % domain_name)
def domain_support(request, domain_name):
return Response("%s support." % domain_name)
def deletion_policy(request):
return Response("Deletion policy.")
@login_required
def domain_start(request, domain_name):
return Response("%s start.\n%s" % (domain_name, request.user.raw_user_data))
</pre><p>A decorator 'login_required' takes care of OpenID/OAuth stuff, so you don't need to do anything at all. The user information is in request.user.raw_user_data.<br />
<p>Let's deploy it to appspot.<br />
<pre>$ python manage.py appcfg update</pre><p>You can check the application with visiting https://marketplace-app.appspot.com/a/example.com/start where example.com should be replaced with your actual domain. This domain must be registered with Google Apps and make sure that "Federated Login using OpenID" is turned on.<br />
<p>If the app is successfully deployed, you can see an OpenID authentication display. Let's move on.<br />
<h1>4. Add your app to your domain</h1><p>Go back to your Vendor Profile Page, and click your application title, then you will see the preview page of your application.<br />
<p>Let's click 'Add it now' button on the right, and enter your google apps domain, and click 'Go'. <br />
Follow an adding application wizard like 1) Agree to terms 2) Grant data access 3) External configuration 4) Enable the app, and wait for a while, and you can see a universal navigation link 'Sample' on the top left corner of your google apps applications.<br />
<p>If you click the link, you can silently signed into your application because Marketplace apps are whitelisted with a particular openid_realm.<br />
<br />
<p>In part 2 of this article, I'll show you how to access user's calendar data with gdata library.<br />
<br />
<p>To be continued..tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com4tag:blogger.com,1999:blog-2144470051777636547.post-8313830390449175732009-12-24T22:18:00.003+09:002009-12-24T22:52:09.366+09:00Integrating websockets with appengine applicationsToday, I've been struggling with an experimental implementation for a pseudo server push on appengine application. So let me share it with you.<br />
<br />
The only problem with appengine is that we can not utilize comet like capabilities on it because of its 30 seconds request limit.<br />
<br />
In this article, I use external websockets server for a pseudo server push on appengine. Here is the diagram(Click for bigger image).<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKQfHOsBP9DG0GDBBlwG9OBVtRQS7L3PcjEQIQIkc76tSWLaiMhQB836U82jvyNOjf787jCGE2lhyD50BKErD4LNUp9njHfTlSFLrzkoes8f2Ip6LcOzU-LVnU15p0iQrpK9wdIQS5O9c/s1600-h/appengine-websockets-integration-diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKQfHOsBP9DG0GDBBlwG9OBVtRQS7L3PcjEQIQIkc76tSWLaiMhQB836U82jvyNOjf787jCGE2lhyD50BKErD4LNUp9njHfTlSFLrzkoes8f2Ip6LcOzU-LVnU15p0iQrpK9wdIQS5O9c/s640/appengine-websockets-integration-diagram.png" /></a><br />
</div><div class="separator" style="clear: both; text-align: left;"><br />
</div><div class="separator" style="clear: both; text-align: left;">Let me explain this diagram a bit.<br />
</div><br />
<ol><li>When a client request for the content, </li>
<li>appengine webapp will returns the newest contents with javascripts for establishing a new websockets connection to an external websockets server.</li>
<li>The browser opens a new websockets connection to the websockets server. This connection will remain as long as possible for notifying updates of the contents.</li>
<li>Another browser requests for updating the contents(e.g. Posting a new comment...etc...).</li>
<li>appengine webapp will save the state in the datastore, and give the newest contents to the client, notifying about this updates to the websockets server as well, simultaneously.</li>
<li>On the websockets server, every server process will receive the notification, and tell their clients that there is some update via the persistent websockets connection.</li>
<li>Now, the first browser knows that there is updated contents on the server, so (in this case) it makes an ajax request to the appengine webapp, and receives the newest contents.</li>
</ol><div>I've implemented <a href="http://websocketchat.appspot.com/">a simple chat with this architecture</a>. Please visit and try it if you'd like. I've tested it only with Chrome 4 or higher(including chromium).<br />
</div><div><br />
</div><div>Now, let's walk through into the code. On the appengine side, when a new comment arives, I need to notify it to the websockets server, so I use urlfetch for this. Here is the code:<br />
</div><div><br />
</div><pre class="prettyprint">def index(request):
form = CommentForm()
if request.method == 'POST':
if form.validate(request.form):
if request.user.is_authenticated():
form.save(owner=request.user)
else:
form.save()
import time
urlfetch.fetch('http://mars.shehas.net/update_webhook.cgi?'
+ str(time.time()))
return redirect(url_for('chat/index'))
comments = Comment.all().order('-created').fetch(20)
return render_to_response('chat/index.html',
{'form': form.as_widget(),
'comments': comments})</pre><br />
The most important thing is that after a new comment is saved, the handler makes an urlfetch call to the external websockets server for notification. It is also important to add time.time() string representation to the url because without this, appengine urlfetch server may cache the response, and this urlfetch call will be useless as a webhook.<br />
<br />
On the client side, we have to create a websocket connection, and set appropriate callbacks on some events of the connection. I've wrote a new class for this.<br />
<br />
update_check_socket.js<br />
<pre class="prettyprint">function UpdateCheckSocket(host, port, resource, statusElement, callback) {
this.host = host;
this.port = port;
this.resource = resource;
this.statusElement = statusElement;
this.callback = callback;
this.ws = new WebSocket("ws://"+this.host+":"+this.port+this.resource);
this.ws.onopen = function(e) {
statusElement.innerHTML='Web sockets connected';
};
this.ws.onmessage = function(e) {
var newDiv = document.createElement('div');
newDiv.innerHTML = e.origin + decodeURIComponent(e.data);
statusElement.insertBefore(newDiv, statusElement.firstChild);
if (decodeURIComponent(e.data) == 'UPDATED') {
callback();
}
};
this.ws.onclose = function(e) {
var newDiv = document.createElement('div');
newDiv.innerHTML = 'Web sockets closed';
statusElement.insertBefore(newDiv, statusElement.firstChild);
};
}
function UpdateCheckSocket_send(message) {
if(typeof(message) == 'undefined' || message =='') {
alert('no message...');
return;
}
this.ws.send(encodeURIComponent(message));
}
UpdateCheckSocket.prototype.send = UpdateCheckSocket_send;
</pre><br />
On the main html, there is a callback for retrieving the newest contents. In some cases, the connection will be closed unintentionally because some network routers might delete the NAT table when there has been no data for few minutes. So there is also the code for avoiding this by sending 'NOOP' string to the server periodically.<br />
<br />
Here is <a href="http://paste.shehas.net/show/30/">the code for main html</a>(as long as I'm concerned, blogger could not handle html well).<br />
<br />
Ok. Let's go to the websockets side. On the external websockets server, I need to 1) accept update notification from appengine webapp(webhook handler), 2) handle websockets connection and notify the update to the client.<br />
<br />
Here is the code for the webhook.<br />
update_webhook.cgi<br />
<pre class="prettyprint">#!/usr/bin/python
import sqlite3
import time
import sys
conn = sqlite3.connect("/tmp/test")
c = conn.cursor()
try:
c.execute('create table updated (t real)')
c.execute('insert into updated values (?)', (time.time(),))
except Exception, e:
sys.stderr.write(str(e))
c.execute('update updated set t = ?', (time.time(),))
conn.commit()
c.close()
conn.close()
print "Content-type: text/plain\n"
print "OK"</pre><br />
This is a very simple script. Its an experimental implementation, so currently it does't check if the request really came from a particular appengine webapp. So do not use this code as is in any production environment.<br />
<br />
The last piece is websocket wsh script(I used <a href="http://code.google.com/p/pywebsocket/">pywebsockets</a> here).<br />
<pre class="prettyprint">import sqlite3
import time
from mod_pywebsocket import msgutil
CHECK_INTERVAL=1
_UPDATE_SIGNAL='UPDATED'
def web_socket_do_extra_handshake(request):
pass # Always accept.
def web_socket_transfer_data(request):
last_checked = time.time()
conn = sqlite3.connect("/tmp/test")
c = conn.cursor()
while True:
time.sleep(CHECK_INTERVAL)
c.execute('select t from updated')
r = c.fetchone()
if r and r[0] > last_checked:
last_checked = r[0]
msgutil.send_message(request, _UPDATE_SIGNAL)</pre><br />
Here, I use sqlite3 for recording the last update time. Using sqlite3 might not be appropriate for the production environment either, again, this is just an experimental implementaion :-)<br />
<br />
Well that's about it. Actually it works, but I don't think this is the best approach. Maybe current implementation won't scale, it might be somewhat cumbersome to setup all of these complicated stuff. I hope I can make these set of code more sophisticated and general in the future, or I hope someone can write better code/architecture for similar purpose.<br />
<br />
Merry X'mas and happy new year :-)tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com4tag:blogger.com,1999:blog-2144470051777636547.post-42519284156480044192009-12-12T08:42:00.003+09:002009-12-12T15:13:12.748+09:00Using appstats with KayGuido <a href="http://groups.google.com/group/google-appengine-python/browse_thread/thread/bff473631cd21159?pli=1">announced the preview release of appstats</a>. This is a tool for visualizing call stats of RPC Calls on appengine that is very useful for improving application's performance.<br />
<br />
This tool consists of two parts; recording part and ui part. There are two ways for configuring the recording part. The first one is to use django middleware which appstats offers. Another way is to configure WSGI middleware. The former way is much easier, so I tried to utilize this django middleware with Kay.<br />
<br />
I have thought that I could easily utilize this middleware with small modifications because Kay has a middleware mechanism very similar to the django's one. Finally, and fortunately I can use this middleware without any modification. That's what I aimed for by implementing Kay's middleware mechanism as similarly as possible to django's one!<br />
<br />
<pre class="prettyprint">MIDDLEWARE_CLASSES = (
'appstats.recording.AppStatsDjangoMiddleware',
'kay.auth.middleware.AuthenticationMiddleware',
)</pre><br />
Configuring the recording part is done by adding appstats.recording.AppStatsDjangoMiddleware to the MIDDLEWARE_CLASSES as above.<br />
<br />
Next I need to configure ui part. According to <a href="http://docs.google.com/Doc?docid=0AQwBadvrXbuVZGZkM3p0czRfMTlkNjI1M2NjOQ&pli=1">the documentation of appstats</a>, I added an entry to my app.yaml as follows.<br />
<br />
<pre class="prettyprint">- url: /stats.*
script: appstats/ui.py
login: admin</pre><br />
This is nearly perfect, but a handlers for viewing source code didn't work correctly, so I needed to add these lines to the upper part of <strike>appstats/ui.py</strike> appengine_config.py.<br />
<span style="color: #990000;">Updated</span>, thanks Guido!<br />
<br />
<pre class="prettyprint">import kay
kay.setup_syspath()</pre><br />
That's all. Now I can use appstats with Kay framework. Here are some screenshots.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><b>This is a dashboard.</b><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8gIX8NzTJDthN-8zl31LvryUhtibJR0immDfWbdbLWa9KwhqRD4pGsNR6AY3LGQYDjHogprH2P_-apL2tFuyL2xxHT7O4R6HdnAzgAb6kWUgJkRWx2NG4Kc4KZwMCmEArGFWN4ASuFko/s1600-h/%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%EF%BC%882009-12-12+8.40.28%EF%BC%89.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8gIX8NzTJDthN-8zl31LvryUhtibJR0immDfWbdbLWa9KwhqRD4pGsNR6AY3LGQYDjHogprH2P_-apL2tFuyL2xxHT7O4R6HdnAzgAb6kWUgJkRWx2NG4Kc4KZwMCmEArGFWN4ASuFko/s640/%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%EF%BC%882009-12-12+8.40.28%EF%BC%89.png" /></a><br />
</div><br />
<div class="separator" style="clear: both; text-align: center;"><b>This is a graphical timeline.</b><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrz5fAkv0JHninpTK9WkjI2wPejtrncFhJXe5883YCP2ulis3e2I3XYqkZKhLcYJr9s7AczQUyciV8b3OrlOflh-B3_II2R4drDxV-EXEVs4L8NuQA2KeSCxQzN28eoSsesqWecfB_RF8/s1600-h/%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%EF%BC%882009-12-12+8.39.08%EF%BC%89.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrz5fAkv0JHninpTK9WkjI2wPejtrncFhJXe5883YCP2ulis3e2I3XYqkZKhLcYJr9s7AczQUyciV8b3OrlOflh-B3_II2R4drDxV-EXEVs4L8NuQA2KeSCxQzN28eoSsesqWecfB_RF8/s640/%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88%EF%BC%882009-12-12+8.39.08%EF%BC%89.png" /></a><br />
</div>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-35170547115736409562009-11-02T00:12:00.000+09:002009-11-02T00:12:26.378+09:00pkg_resources の warning を抑制するsitecustomize.py を site-packages 内に下記の内容で作成する<br />
<br />
<pre class="prettyprint">import warnings
warnings.filterwarnings('ignore',
message=r'Module .*? is being added to sys\.path', append=True)</pre><br />
ここに書いてありました:<br />
http://lucumr.pocoo.org/2008/2/19/sick-of-pkg-resources-warningstmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-30488295017858389522009-10-26T09:56:00.001+09:002009-10-26T09:58:27.781+09:00More and more lazy loading...I've spent a little while to make Kay load modules in more lazily manner, then I've got faster result on cold start, so I'd like to share the result with you.<br />
<br />
Basically, I've made this test in the same way as described <a href="http://takashi-matsuo.blogspot.com/2009/10/minimum-cost-of-various-frameworks-cold.html">this entry</a>.<br />
<br />
<h4>Kay(changeset:4a854c31abbc)</h4><pre>10-25 05:33PM 31.791 / 200 344ms 544cpu_ms
10-25 05:34PM 07.998 / 200 354ms 544cpu_ms
10-25 05:34PM 37.308 / 200 355ms 563cpu_ms
10-25 05:35PM 00.452 / 200 336ms 525cpu_ms
10-25 05:35PM 27.638 / 200 332ms 525cpu_ms
10-25 05:40PM 45.837 / 200 365ms 544cpu_ms
</pre><br />
Now Kay gets significant faster result! I'm planning to release Kay-0.3.0 soon.tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-8267195846821901512009-10-21T02:45:00.011+09:002009-10-21T08:09:21.591+09:00Minimum cost for warming-up various frameworks(and more)<h2>Overview</h2><br />
There are lots of frameworks that works on appengine, so people might wonder which framework is the best one for them. So, I did a simple benchmark to see how much cpu time to warm-up various frameworks for appengine with minimized applications.<br />
<br />
<h2>How to test</h2>The rule is quite simple.<br />
<ul><li>Use template system for rendering<br />
<li>Use very simple template<br />
<li>No middleware, context_processors<br />
<li>The view passes single string ('hello') to the template<br />
<li>No i18n (USE_I18N=False)<br />
</ul>Today, I tested 3 frameworks; <ul><li>webapp+django template<br />
<li><a href="http://code.google.com/p/kay-framework/">Kay framework</a> repository tip (changeset: 6c871008872a)<br />
<li><a href="http://code.google.com/p/app-engine-patch/">app-engine-patch-1.1RC</a><br />
</ul>The single series of tests was made as follows; <ol><li>Upload an application<br />
<li>Access by browser once<br />
<li>View logs on admin console and record ms and cpu_ms values<br />
<li>Go to next framework in turn and do the same test<br />
</ol><h2>Results</h2>And I did this series of tests five times and got following result: <h3>Cold Starting result</h3><h4>webapp+django template</h4><pre>10-20 09:29AM 32.796 / 200 332ms 369cpu_ms
10-20 09:31AM 45.991 / 200 278ms 330cpu_ms
10-20 09:33AM 56.086 / 200 352ms 369cpu_ms
10-20 09:36AM 16.630 / 200 245ms 350cpu_ms
10-20 09:39AM 13.148 / 200 266ms 350cpu_ms
</pre><h4>Kay</h4><pre>10-20 09:30AM 13.555 / 200 504ms 700cpu_ms
10-20 09:32AM 27.076 / 200 436ms 641cpu_ms
10-20 09:34AM 37.841 / 200 417ms 621cpu_ms
10-20 09:37AM 01.469 / 200 471ms 641cpu_ms
10-20 09:41AM 40.874 / 200 454ms 660cpu_ms
</pre><h4>app-engine-patch</h4><pre>10-20 09:31AM 00.640 / 200 663ms 1010cpu_ms
10-20 09:33AM 12.117 / 200 594ms 991cpu_ms
10-20 09:35AM 29.921 / 200 654ms 1030cpu_ms
10-20 09:38AM 00.888 / 200 629ms 1030cpu_ms
10-20 09:46AM 07.109 / 200 702ms 1108cpu_ms
</pre>Additionally, I did some tests for serving by hot instances: <h3>Served by hot instances result</h3><h4>webapp+django template</h4><pre>10-20 09:39AM 23.025 / 200 6ms 3cpu_ms
10-20 09:39AM 24.104 / 200 6ms 5cpu_ms
10-20 09:39AM 25.212 / 200 9ms 5cpu_ms
10-20 09:39AM 26.310 / 200 6ms 4cpu_ms
10-20 09:39AM 27.414 / 200 7ms 4cpu_ms
</pre><h4>Kay</h4><pre>10-20 09:41AM 42.991 / 200 13ms 5cpu_ms
10-20 09:41AM 45.691 / 200 6ms 5cpu_ms
10-20 09:41AM 46.538 / 200 7ms 5cpu_ms
10-20 09:41AM 47.506 / 200 9ms 8cpu_ms
10-20 09:41AM 48.536 / 200 8ms 5cpu_ms
</pre><h4>app-engine-patch</h4><pre>10-20 09:46AM 09.021 / 200 10ms 10cpu_ms
10-20 09:46AM 09.949 / 200 8ms 8cpu_ms
10-20 09:46AM 10.869 / 200 8ms 6cpu_ms
10-20 09:46AM 11.834 / 200 11ms 11cpu_ms
10-20 09:46AM 12.743 / 200 9ms 10cpu_ms
</pre><h2>The application used in these tests</h2><p>The template used is common among these frameworks.</p>index.html: <pre class="prettyprint"><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"%gt;
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Top Page - myapp</title>
</head>
<body>
{{ message }}
</body>
</html>
</pre>Here are the application settings for each frameworks: <h3>webapp+django template</h3>main.py: <pre class="prettyprint">#!/usr/bin/env python
#
# Copyright 2007 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import logging
import os
import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.ext import db
from google.appengine.ext.webapp import template
class MainHandler(webapp.RequestHandler):
def get(self):
contents = {'message': 'hello'}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, contents))
def main():
logging.getLogger().setLevel(logging.debug)
application = webapp.WSGIApplication([('/', MainHandler)],
debug=True)
wsgiref.handlers.CGIHandler().run(application)
if __name__ == '__main__':
main()
</pre><h3>Kay</h3>settings.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
"""
A sample of kay settings.
:Copyright: (c) 2009 Accense Technology, Inc.
Takashi Matsuo <tmatsuo@candit.jp>,
All rights reserved.
:license: BSD, see LICENSE for more details.
"""
DEFAULT_TIMEZONE = 'Asia/Tokyo'
DEBUG = False
SECRET_KEY = 'ReplaceItWithSecretString'
SESSION_PREFIX = 'gaesess:'
COOKIE_AGE = 1209600 # 2 weeks
COOKIE_NAME = 'KAY_SESSION'
ADD_APP_PREFIX_TO_KIND = True
ADMINS = (
)
TEMPLATE_DIRS = (
)
USE_I18N = False
DEFAULT_LANG = 'en'
INSTALLED_APPS = (
'myapp',
)
APP_MOUNT_POINTS = {
'myapp': '/',
}
CONTEXT_PROCESSORS = (
)
JINJA2_FILTERS = {
}
MIDDLEWARE_CLASSES = (
)
AUTH_USER_BACKEND = 'kay.auth.backend.GoogleBackend'
AUTH_USER_MODEL = 'kay.auth.models.GoogleUser'
</pre>myapp/urls.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
# myapp.urls
from werkzeug.routing import (
Map, Rule, Submount,
EndpointPrefix, RuleTemplate,
)
import myapp.views
def make_rules():
return [
EndpointPrefix('myapp/', [
Rule('/', endpoint='index'),
]),
]
all_views = {
'myapp/index': myapp.views.index,
}
</pre>myapp/views.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
# myapp.views
from kay.utils import render_to_response
# Create your views here.
def index(request):
return render_to_response('myapp/index.html', {'message': 'Hello'})
</pre><h3>app-engine-patch</h3>settings.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
from ragendja.settings_pre import *
# Increase this when you update your media on the production site, so users
# don't have to refresh their cache. By setting this your MEDIA_URL
# automatically becomes /media/MEDIA_VERSION/
MEDIA_VERSION = 1
# By hosting media on a different domain we can get a speedup (more parallel
# browser connections).
#if on_production_server or not have_appserver:
# MEDIA_URL = 'http://media.mydomain.com/media/%d/'
# Add base media (jquery can be easily added via INSTALLED_APPS)
COMBINE_MEDIA = {
'combined-%(LANGUAGE_CODE)s.js': (
# See documentation why site_data can be useful:
# http://code.google.com/p/app-engine-patch/wiki/MediaGenerator
'.site_data.js',
),
'combined-%(LANGUAGE_DIR)s.css': (
'global/look.css',
),
}
# Change your email settings
if on_production_server:
DEFAULT_FROM_EMAIL = 'bla@bla.com'
SERVER_EMAIL = DEFAULT_FROM_EMAIL
# Make this unique, and don't share it with anybody.
SECRET_KEY = '1234567890'
#ENABLE_PROFILER = True
#ONLY_FORCED_PROFILE = True
#PROFILE_PERCENTAGE = 25
#SORT_PROFILE_RESULTS_BY = 'cumulative' # default is 'time'
# Profile only datastore calls
#PROFILE_PATTERN = 'ext.db..+\((?:get|get_by_key_name|fetch|count|put)\)'
# Enable I18N and set default language to 'en'
USE_I18N = False
LANGUAGE_CODE = 'en'
# Restrict supported languages (and JS media generation)
LANGUAGES = (
('de', 'German'),
('en', 'English'),
)
TEMPLATE_CONTEXT_PROCESSORS = (
)
MIDDLEWARE_CLASSES = (
)
# Google authentication
AUTH_USER_MODULE = 'ragendja.auth.google_models'
AUTH_ADMIN_MODULE = 'ragendja.auth.google_admin'
# Hybrid Django/Google authentication
#AUTH_USER_MODULE = 'ragendja.auth.hybrid_models'
LOGIN_URL = '/account/login/'
LOGOUT_URL = '/account/logout/'
LOGIN_REDIRECT_URL = '/'
INSTALLED_APPS = (
# Add jquery support (app is in "common" folder). This automatically
# adds jquery to your COMBINE_MEDIA['combined-%(LANGUAGE_CODE)s.js']
# Note: the order of your INSTALLED_APPS specifies the order in which
# your app-specific media files get combined, so jquery should normally
# come first.
'appenginepatcher',
'ragendja',
'myapp',
)
# List apps which should be left out from app settings and urlsauto loading
IGNORE_APP_SETTINGS = IGNORE_APP_URLSAUTO = (
# Example:
# 'django.contrib.admin',
# 'django.contrib.auth',
# 'yetanotherapp',
)
# Remote access to production server (e.g., via manage.py shell --remote)
DATABASE_OPTIONS = {
# Override remoteapi handler's path (default: '/remote_api').
# This is a good idea, so you make it not too easy for hackers. ;)
# Don't forget to also update your app.yaml!
#'remote_url': '/remote-secret-url',
# !!!Normally, the following settings should not be used!!!
# Always use remoteapi (no need to add manage.py --remote option)
#'use_remote': True,
# Change appid for remote connection (by default it's the same as in
# your app.yaml)
#'remote_id': 'otherappid',
# Change domain (default: <remoteid>.appspot.com)
#'remote_host': 'bla.com',
}
from ragendja.settings_post import *
</pre>urls.py(minimized): <pre class="prettyprint"># -*- coding: utf-8 -*-
from django.conf.urls.defaults import *
from ragendja.urlsauto import urlpatterns
from ragendja.auth.urls import urlpatterns as auth_patterns
urlpatterns = auth_patterns + patterns('',
(r'', include('myapp.urls')),
) + urlpatterns
</pre>myapp/urls.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
from django.conf.urls.defaults import *
urlpatterns = patterns(
'myapp.views',
url(r'^$', 'index', name='myapp_index'),
)
</pre>myapp/views.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
from ragendja.template import render_to_response
def index(request):
return render_to_response(request, 'myapp/index.html',
{'message': 'Hello'})
</pre><h2>Conclusion</h2><p>There are significant defferences among warm-up costs of these three frameworks. But when it comes to serving by warm instances, the defferences are rather small. <p>Actually you can choose whatever you want, but what if you must to choose one framework among these three options? <p>webapp+django template is really fast(actually without django template, webapp is much faster than this, but I think its unfair ;-P), so If you really need speed, webapp might be the best option. <p>Having said that, sometimes we need a fancy way for constructing rather complex applications. Perhaps you can choose Kay/app-engine-patch if you can accept 2 or 3 times(obviously compared with webapp) costs on cold-startup. <p>If you love Django admin capability and hardly throw it away, only app-engine-patch could be your option. <p>Actually I don't know much about app-engine-patch, so these settings is not enough. Please let me know if that's the case. If you make some tests similar to these tests, I'd be glad to know the result. <div style="color: red">The result with <a href="http://code.google.com/p/tipfy/">tipfy</a> is added.</div><a href="http://code.google.com/p/tipfy/">tipfy</a> is one of the fastest and lightest frameworks. You can use a very nice debugger(werkzeug's) in dev environment(This debugger is also available with Kay).
<h3>Cold Starting result</h3><h4>tipfy</h4><pre>10-20 03:43PM 28.814 / 200 411ms 602cpu_ms
10-20 03:44PM 31.751 / 200 367ms 563cpu_ms
10-20 03:45PM 09.913 / 200 407ms 563cpu_ms
10-20 03:46PM 23.932 / 200 413ms 602cpu_ms
10-20 03:47PM 19.530 / 200 416ms 583cpu_ms
</pre><h3>Served by hot instances result</h3><h4>tipfy</h4><pre>10-20 03:47PM 21.859 / 200 6ms 4cpu_ms
10-20 03:47PM 23.003 / 200 8ms 5cpu_ms
10-20 03:47PM 24.177 / 200 6ms 4cpu_ms
10-20 03:47PM 25.166 / 200 6ms 4cpu_ms
10-20 03:47PM 26.181 / 200 5ms 4cpu_ms
</pre><h3>Application</h3><h4>tipfy</h4>urls.py: <pre class="prettyprint"># -*- coding: utf-8 -*-
"""
urls
~~~~
URL definitions.
:copyright: 2009 by tipfy.org.
:license: BSD, see LICENSE.txt for more details.
"""
from tipfy import Rule
urls = [
Rule('/', endpoint='home', handler='hello:HelloWorldHandler'),
]
</pre>hello.py: <pre class="prettyprint">from tipfy import RequestHandler
from tipfy.ext.jinja2 import render_response
class HelloWorldHandler(RequestHandler):
def get(self, **kwargs):
context = {'message': 'hello'}
return render_response('index.html', **context)
</pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com15tag:blogger.com,1999:blog-2144470051777636547.post-54816677475520794552009-08-31T22:30:00.002+09:002009-08-31T22:38:42.342+09:00Using pyjamas on KayI use ${PYJAMAS_HOME} and ${PROJECT_DIR} variable for explanation here. ${PYJAMAS_HOME} is a directory in which you installed pyjamas. ${PROJECT_DIR} is your project directory.<br /><br />For opener, please begin with customizing ${PYJAMAS_HOME}/examples/jsonrpc. You need only JSONRPCExample.html in the `public` directory.<br /><br /><pre class="prettyprint"><br />$ tree kay-pyjamas-sample<br />kay-pyjamas-sample<br />|-- JSONRPCExample.py<br />`-- public<br /> `-- JSONRPCExample.html<br /><br />1 directory, 2 files<br /></pre><br /><br />Edit your own JSONRPCExample.py as follows:<br /><pre class="prettyprint"><br />import pyjd # dummy in pyjs<br /><br />from pyjamas.ui.RootPanel import RootPanel<br />from pyjamas.ui.TextArea import TextArea<br />from pyjamas.ui.Label import Label<br />from pyjamas.ui.Button import Button<br />from pyjamas.ui.HTML import HTML<br />from pyjamas.ui.VerticalPanel import VerticalPanel<br />from pyjamas.ui.HorizontalPanel import HorizontalPanel<br />from pyjamas.ui.ListBox import ListBox<br />from pyjamas.JSONService import JSONProxy<br /><br />class JSONRPCExample:<br /> def onModuleLoad(self):<br /> self.TEXT_WAITING = "Waiting for response..."<br /> self.TEXT_ERROR = "Server Error"<br /> self.METHOD_ECHO = "Echo"<br /> self.METHOD_REVERSE = "Reverse"<br /> self.METHOD_UPPERCASE = "UPPERCASE"<br /> self.METHOD_LOWERCASE = "lowercase"<br /> self.methods = [self.METHOD_ECHO, self.METHOD_REVERSE, self.METHOD_UPPERCASE, self.METHOD_LOWERCASE]<br /><br /> self.remote_py = EchoServicePython()<br /><br /> self.status=Label()<br /> self.text_area = TextArea()<br /> self.text_area.setText("""{'Test'} [\"String\"]<br />\tTest Tab<br />Test Newline\n<br />after newline<br />""" + r"""Literal String:<br />{'Test'} [\"String\"]<br />""")<br /> self.text_area.setCharacterWidth(80)<br /> self.text_area.setVisibleLines(8)<br /> <br /> self.method_list = ListBox()<br /> self.method_list.setName("hello")<br /> self.method_list.setVisibleItemCount(1)<br /> for method in self.methods:<br /> self.method_list.addItem(method)<br /> self.method_list.setSelectedIndex(0)<br /><br /> method_panel = HorizontalPanel()<br /> method_panel.add(HTML("Remote string method to call: "))<br /> method_panel.add(self.method_list)<br /> method_panel.setSpacing(8)<br /><br /> self.button_py = Button("Send to Python Service", self)<br /><br /> buttons = HorizontalPanel()<br /> buttons.add(self.button_py)<br /> buttons.setSpacing(8)<br /> <br /> info = """<h2>JSON-RPC Example</h2><br /> <p>This example demonstrates the calling of server services with<br /> <a href="http://json-rpc.org/">JSON-RPC</a>.<br /> </p><br /> <p>Enter some text below, and press a button to send the text<br /> to an Echo service on your server. An echo service simply sends the exact same text back that it receives.<br /> </p>"""<br /> <br /> panel = VerticalPanel()<br /> panel.add(HTML(info))<br /> panel.add(self.text_area)<br /> panel.add(method_panel)<br /> panel.add(buttons)<br /> panel.add(self.status)<br /> <br /> RootPanel().add(panel)<br /><br /> def onClick(self, sender):<br /> self.status.setText(self.TEXT_WAITING)<br /> method = self.methods[self.method_list.getSelectedIndex()]<br /> text = self.text_area.getText()<br /> id = -1<br /> if method == self.METHOD_ECHO:<br /> id = self.remote_py.echo(text, self)<br /> elif method == self.METHOD_REVERSE:<br /> id = self.remote_py.reverse(text, self)<br /> elif method == self.METHOD_UPPERCASE:<br /> id = self.remote_py.uppercase(text, self)<br /> elif method == self.METHOD_LOWERCASE:<br /> id = self.remote_py.lowercase(text, self)<br /> if id<0:<br /> self.status.setText(self.TEXT_ERROR)<br /><br /> def onRemoteResponse(self, response, request_info):<br /> self.status.setText(response)<br /><br /> def onRemoteError(self, code, message, request_info):<br /> self.status.setText("Server Error or Invalid Response: ERROR %d - %s" %<br /> (code, message))<br /><br /><br />class EchoServicePython(JSONProxy):<br /> def __init__(self):<br /> JSONProxy.__init__(self, "/json", ["echo", "reverse", "uppercase", "lowercase"])<br /><br />if __name__ == '__main__':<br /> # for pyjd, set up a web server and load the HTML from there:<br /> # this convinces the browser engine that the AJAX will be loaded<br /> # from the same URI base as the URL, it's all a bit messy...<br /> pyjd.setup("http://127.0.0.1/examples/jsonrpc/public/JSONRPCExample.html")<br /> app = JSONRPCExample()<br /> app.onModuleLoad()<br /> pyjd.run()<br /></pre><br /><br />Then, cd into this directory and compile it. This time, I use ${PROJECT_DIR}/media for a target directory.<br /><br /><pre class="prettyprint"><br />$ cd kay-pyjamas-sample<br />$ ${PYJAMAS_HOME}/bin/pyjsbuild -o ${PROJECT_DIR}/media JSONRPCExample.py<br /></pre><br /><br />If succeed, you'll get ${PROJECT_DIR}/media/JSONRPCExample.html. Please access http://localhost:8080/media/JSONRPCExample.html and, you'll get forms for testing jsonrpc. <br /><br />Let's write server side code. First, you need to edit urls.py in your application.<br /><br />${PROJECT_DIR}/myapp/urls.py<br /><pre class="prettyprint"><br /># -*- coding: utf-8 -*-<br /># myapp.urls<br /><br /><br />from werkzeug.routing import (<br /> Map, Rule, Submount,<br /> EndpointPrefix, RuleTemplate,<br />)<br />import myapp.views<br /><br />def make_rules():<br /> return [<br /> EndpointPrefix('myapp/', [<br /> Rule('/', endpoint='index'),<br /> Rule('/json', endpoint='json_rpc'),<br /> ]),<br /> ]<br /><br />all_views = {<br /> 'myapp/index': myapp.views.index,<br /> 'myapp/json_rpc': myapp.views.json_rpc,<br />}<br /></pre><br /><br />You also need to edit views.py.<br /><br />${PROJECT_DIR}/myapp/views.py<br /><pre class="prettyprint"><br /># -*- coding: utf-8 -*-<br /># myapp.views<br /># ... (snip)<br /><br />import simplejson<br /><br /># ... (snip)<br /><br />def json_uppercase(args):<br /> return [args[0].upper()]<br /><br />def json_echo(args):<br /> return [args[0]]<br /><br />def json_reverse(args):<br /> return [args[0][::-1]]<br /><br />def json_lowercase(args):<br /> return [args[0].lower()]<br /><br />def json_rpc(request):<br /> if request.method:<br /> args = simplejson.loads(request.data)<br /> method_name = 'json_%s' % args[u"method"]<br /> json_func = globals().get(method_name)<br /> json_params = args[u"params"]<br /> json_method_id = args[u"id"]<br /> result = json_func(json_params)<br /> args.pop(u"method")<br /> args["result"] = result[0]<br /> args["error"] = None<br /> return Response(simplejson.dumps(args), content_type="application/json")<br /> else:<br /> return Response('Error')<br /><br /></pre><br /><br />That's all. Please enjoy :-)tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com1tag:blogger.com,1999:blog-2144470051777636547.post-28302625892543101742009-08-31T22:04:00.002+09:002009-08-31T22:23:50.797+09:00Kay で pyjamas を使う<a href="http://kay-docs-jp.shehas.net/">日本語の情報もできた</a>のでちょっと変わり種をやります。<br /><br />Python 版 GWT である <a href="http://pyjs.org/">pyjamas</a> を Kay と組み合わせてみました.. と言っても jsonrpc の簡単なサンプルを動かしただけですが。<br /><br />Pyjamas をインストールした場所を ${PYJAMAS_HOME} と、プロジェクトのディレクトリを ${PROJECT_DIR} として進めましょう。<br /><br />${PYJAMAS_HOME}/examples/jsonrpc をいじって使います。public 以下は JSONRPCExample.html だけ必要みたい。<br /><br /><pre class="prettyprint"><br />$ tree kay-pyjamas-sample<br />kay-pyjamas-sample<br />|-- JSONRPCExample.py<br />`-- public<br /> `-- JSONRPCExample.html<br /><br />1 directory, 2 files<br /></pre><br /><br />JSONRPCExample.py を下記のように少しいじります。<br /><pre class="prettyprint"><br />import pyjd # dummy in pyjs<br /><br />from pyjamas.ui.RootPanel import RootPanel<br />from pyjamas.ui.TextArea import TextArea<br />from pyjamas.ui.Label import Label<br />from pyjamas.ui.Button import Button<br />from pyjamas.ui.HTML import HTML<br />from pyjamas.ui.VerticalPanel import VerticalPanel<br />from pyjamas.ui.HorizontalPanel import HorizontalPanel<br />from pyjamas.ui.ListBox import ListBox<br />from pyjamas.JSONService import JSONProxy<br /><br />class JSONRPCExample:<br /> def onModuleLoad(self):<br /> self.TEXT_WAITING = "Waiting for response..."<br /> self.TEXT_ERROR = "Server Error"<br /> self.METHOD_ECHO = "Echo"<br /> self.METHOD_REVERSE = "Reverse"<br /> self.METHOD_UPPERCASE = "UPPERCASE"<br /> self.METHOD_LOWERCASE = "lowercase"<br /> self.methods = [self.METHOD_ECHO, self.METHOD_REVERSE, self.METHOD_UPPERCASE, self.METHOD_LOWERCASE]<br /><br /> self.remote_py = EchoServicePython()<br /><br /> self.status=Label()<br /> self.text_area = TextArea()<br /> self.text_area.setText("""{'Test'} [\"String\"]<br />\tTest Tab<br />Test Newline\n<br />after newline<br />""" + r"""Literal String:<br />{'Test'} [\"String\"]<br />""")<br /> self.text_area.setCharacterWidth(80)<br /> self.text_area.setVisibleLines(8)<br /> <br /> self.method_list = ListBox()<br /> self.method_list.setName("hello")<br /> self.method_list.setVisibleItemCount(1)<br /> for method in self.methods:<br /> self.method_list.addItem(method)<br /> self.method_list.setSelectedIndex(0)<br /><br /> method_panel = HorizontalPanel()<br /> method_panel.add(HTML("Remote string method to call: "))<br /> method_panel.add(self.method_list)<br /> method_panel.setSpacing(8)<br /><br /> self.button_py = Button("Send to Python Service", self)<br /><br /> buttons = HorizontalPanel()<br /> buttons.add(self.button_py)<br /> buttons.setSpacing(8)<br /> <br /> info = """<h2>JSON-RPC Example</h2><br /> <p>This example demonstrates the calling of server services with<br /> <a href="http://json-rpc.org/">JSON-RPC</a>.<br /> </p><br /> <p>Enter some text below, and press a button to send the text<br /> to an Echo service on your server. An echo service simply sends the exact same text back that it receives.<br /> </p>"""<br /> <br /> panel = VerticalPanel()<br /> panel.add(HTML(info))<br /> panel.add(self.text_area)<br /> panel.add(method_panel)<br /> panel.add(buttons)<br /> panel.add(self.status)<br /> <br /> RootPanel().add(panel)<br /><br /> def onClick(self, sender):<br /> self.status.setText(self.TEXT_WAITING)<br /> method = self.methods[self.method_list.getSelectedIndex()]<br /> text = self.text_area.getText()<br /> id = -1<br /> if method == self.METHOD_ECHO:<br /> id = self.remote_py.echo(text, self)<br /> elif method == self.METHOD_REVERSE:<br /> id = self.remote_py.reverse(text, self)<br /> elif method == self.METHOD_UPPERCASE:<br /> id = self.remote_py.uppercase(text, self)<br /> elif method == self.METHOD_LOWERCASE:<br /> id = self.remote_py.lowercase(text, self)<br /> if id<0:<br /> self.status.setText(self.TEXT_ERROR)<br /><br /> def onRemoteResponse(self, response, request_info):<br /> self.status.setText(response)<br /><br /> def onRemoteError(self, code, message, request_info):<br /> self.status.setText("Server Error or Invalid Response: ERROR %d - %s" %<br /> (code, message))<br /><br /><br />class EchoServicePython(JSONProxy):<br /> def __init__(self):<br /> JSONProxy.__init__(self, "/json", ["echo", "reverse", "uppercase", "lowercase"])<br /><br />if __name__ == '__main__':<br /> # for pyjd, set up a web server and load the HTML from there:<br /> # this convinces the browser engine that the AJAX will be loaded<br /> # from the same URI base as the URL, it's all a bit messy...<br /> pyjd.setup("http://127.0.0.1/examples/jsonrpc/public/JSONRPCExample.html")<br /> app = JSONRPCExample()<br /> app.onModuleLoad()<br /> pyjd.run()<br /></pre><br /><br />そしたらそのディレクトリに入ってコンパイルします。今回はターゲットを直<br />接 ${PROJECT_DIR}/media にしてしまいます。<br /><br /><pre class="prettyprint"><br />$ cd kay-pyjamas-sample<br />$ ${PYJAMAS_HOME}/bin/pyjsbuild -o ${PROJECT_DIR}/media JSONRPCExample.py<br /></pre><br /><br />成功すると ${PROJECT_DIR}/media/JSONRPCExample.html が出来ます。<br />http://localhost:8080/media/JSONRPCExample.html で画面が出ていればまあ成功です。<br /><br />サーバーサイドは下記の通りにします。myapp はアプリケーション名に読み換<br />えてくださいね。<br /><br />${PROJECT_DIR}/myapp/urls.py<br /><pre class="prettyprint"><br /># -*- coding: utf-8 -*-<br /># myapp.urls<br /><br /><br />from werkzeug.routing import (<br /> Map, Rule, Submount,<br /> EndpointPrefix, RuleTemplate,<br />)<br />import myapp.views<br /><br />def make_rules():<br /> return [<br /> EndpointPrefix('myapp/', [<br /> Rule('/', endpoint='index'),<br /> Rule('/json', endpoint='json_rpc'),<br /> ]),<br /> ]<br /><br />all_views = {<br /> 'myapp/index': myapp.views.index,<br /> 'myapp/json_rpc': myapp.views.json_rpc,<br />}<br /></pre><br /><br />最後に view です。<br /><br />${PROJECT_DIR}/myapp/views.py<br /><pre class="prettyprint"><br /># -*- coding: utf-8 -*-<br /># myapp.views<br /># ... 省略<br /><br />import simplejson<br /><br /># ... 省略<br /><br />def json_uppercase(args):<br /> return [args[0].upper()]<br /><br />def json_echo(args):<br /> return [args[0]]<br /><br />def json_reverse(args):<br /> return [args[0][::-1]]<br /><br />def json_lowercase(args):<br /> return [args[0].lower()]<br /><br />def json_rpc(request):<br /> if request.method:<br /> args = simplejson.loads(request.data)<br /> method_name = 'json_%s' % args[u"method"]<br /> json_func = globals().get(method_name)<br /> json_params = args[u"params"]<br /> json_method_id = args[u"id"]<br /> result = json_func(json_params)<br /> args.pop(u"method")<br /> args["result"] = result[0]<br /> args["error"] = None<br /> return Response(simplejson.dumps(args), content_type="application/json")<br /> else:<br /> return Response('Error')<br /><br /></pre><br /><br />適当な作りですが、まあこれで動きます。興味があれば色々工夫してみてくだ<br />さい。<br /><br />今日はここでおしまいです。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-79153714803048737122009-08-04T14:24:00.010+09:002009-08-04T15:10:13.047+09:00Kay framework を使ってみる<a href="http://code.google.com/p/kay-framework/">Kay framework</a> を 7/7 にリリースしたわけですが、日本語での情報が無いので少しずつ書いていくことにしようと思います。<br /><br /><h2>始め方</h2><br />必要なものは下記の通りです。<br /><ul><br /><li>Python-2.5<br /><li>App Engine SDK Python<br /><li>Kay framework<br /></ul><br /><br />macports を使って python25 を入れた場合は、他に下記もインストールしましょう。<br /><ul><br /><li>py25-hashlib<br /><li>py25-socket-ssl<br /><li>py25-pil<br /><li>py25-ipython<br /></ul><br /><br />Kay のリリース版を使う場合は、下記のダウンロードページから tar ball を落として使います。リポジトリを追いかける場合は、mercurial で clone してください。<br /><br />ダウンロードページ: <a href="http://code.google.com/p/kay-framework/downloads/list">http://code.google.com/p/kay-framework/downloads/list</a><br /><br />clone するには:<br /><pre class="prettyprint"><br />$ hg clone https://kay-framework.googlecode.com/hg/ kay<br /></pre><br /><br /><h2>プロジェクトを始める</h2><br />kay の manage.py スクリプトでプロジェクトディレクトリを作る事ができます。<br /><pre class="prettyprint"><br />$ python kay/manage.py startproject myproject<br />$ tree myproject<br />myproject<br />|-- app.yaml<br />|-- kay -> /Users/tmatsuo/work/tmp/kay/kay<br />|-- manage.py -> /Users/tmatsuo/work/tmp/kay/manage.py<br />|-- settings.py<br />`-- urls.py<br /><br />1 directory, 4 files<br /></pre><br /><br />シンボリックリンクをサポートしているプラットフォームでは kay ディレクトリと manage.py へのシンボリックリンクが作成されます。後で kay の場所を動かすときっと動かなくなるのですが、そんな時はリンクを張り直してください。<br /><br /><h2>アプリケーションを作る</h2><br />出来たばかりの myproject ディレクトリに cd して、早速アプリケーションを作りましょう。<br /><pre class="prettyprint"><br />$ cd myproject<br />$ python manage.py startapp myapp<br />$ tree myapp<br />myapp<br />|-- __init__.py<br />|-- models.py<br />|-- templates<br />| `-- index.html<br />|-- urls.py<br />`-- views.py<br /><br />1 directory, 5 files<br /></pre><br /><br />こうして作成した myapp を動作させるにはもうひと手間必要です。settings.py の INSTALLED_APPS に登録します。必要なら APP_MOUNT_POINTS も登録します。下記の例では、アプリケーションをルート URL にマウントする例です。APP_MOUNT_POINTS を設定しない場合は、/myapp というようにアプリケーション名 URL にマウントされます。<br /><br />settings.py<br /><pre><br />#$/usr/bin/python<br />#..<br />#..<br /><br />INSTALLED_APPS = (<br /> 'kay.sessions',<br /> 'myapp',<br />)<br /><br />APP_MOUNT_POINTS = {<br /> 'myapp': '/',<br />}<br /></pre><br /><br />見れば分かると思いますが、INSTALLED_APPS はタプルで、APP_MOUNT_POINTS は dict になっています。<br /><br /><h2>アプリケーションを動かす</h2><br />作ったアプリケーションを動かしてみましょう。下記のコマンドで開発サーバが起動する筈です。<br /><pre class="prettyprint"><br />$ python manage.py runserver<br />INFO 2009-08-04 05:48:21,339 appengine_rpc.py:157] Server: appengine.google.com<br />...<br />...<br />INFO 2009-08-04 05:48:21,448 dev_appserver_main.py:465] Running application myproject on port 8080: http://localhost:8080<br /></pre><br />この状態で http://localhost:8080/ にアクセスすると、「Hello」又は「こんにちは」と表示される筈です。<br /><br /><h2>GAE にアップロードする</h2><br />GAE にアップロードするには、対象の appid を app.yaml の application に設定してから、下記のコマンドを使用します。<br /><pre class="prettyprint"><br />$ python manage.py appcfg update<br /></pre><br /><br />成功すると、http://your-appid.appspot.com/ でアクセスできるようになります。<br /><br />今日はここまでにしますね。けっこう簡単に始められる事がお分かりいただけたでしょうか。次回はモデル定義とか、フォームの自動生成などについてお話しする予定です。<br /><br />当分このシリーズを続けるつもりなので、取り上げて欲しいトピックなどがあればコメントください。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com3tag:blogger.com,1999:blog-2144470051777636547.post-56041707004628386172009-05-15T00:45:00.003+09:002009-05-15T10:02:54.633+09:00A patch for Werkzeug debugger on GAEI wrote about <a href="http://takashi-matsuo.blogspot.com/2009/05/using-werkzeugs-debugger-on-gae-dev.html">using werkzeug on gae</a>, but actually there are still several problems in debug console. So I wrote a patch for it.<br /><br /><pre class="prettyprint"><br /><br />diff -r 8ffe4e637e17 werkzeug/debug/console.py<br />--- a/werkzeug/debug/console.py Sat Apr 25 17:06:40 2009 +0000<br />+++ b/werkzeug/debug/console.py Fri May 15 00:40:32 2009 +0900<br />@@ -32,6 +32,12 @@<br /> def close(self):<br /> pass<br /> <br />+ def flush(self):<br />+ pass<br />+<br />+ def seek(self, n, mode=0):<br />+ pass<br />+<br /> def reset(self):<br /> val = ''.join(self._buffer)<br /> del self._buffer[:]<br />@@ -48,12 +54,18 @@<br /> def writelines(self, x):<br /> self._write(escape(''.join(x)))<br /> <br />+ def readline(self):<br />+ if len(self._buffer) == 0:<br />+ return ''<br />+ ret = self._buffer[0]<br />+ del self._buffer[0]<br />+ return ret<br /> <br /> class ThreadedStream(object):<br /> """Thread-local wrapper for sys.stdout for the interactive console."""<br /> <br /> def push():<br />- if sys.stdout is sys.__stdout__:<br />+ if not isinstance(sys.stdout, ThreadedStream):<br /> sys.stdout = ThreadedStream()<br /> _local.stream = HTMLStringO()<br /> push = staticmethod(push)<br /><br /></pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-66882569950238086482009-05-14T20:13:00.004+09:002009-05-14T20:27:47.693+09:00Using Werkzeug's debugger on GAE dev serverThere is a very good web framework called Werkzeug. The debugger of Werkzeug is pretty nice and we can use inline console on our web browser with it.<br /><br />The author of Werkzeug wrote <a href="http://dev.pocoo.org/projects/werkzeug/wiki/UsingDebuggerWithAppEngine">an article </a> of how to use this debugger with GAE dev server. However according this article, the inline console is not usable.<br /><br />After a little struggling, finally I can use the debugger on GAE dev server. GAE dev server creates an instance of DebuggedApplication on every request. I think its the reason why. So, it seems that specifying the instance as the module global singleton made it work.<br /><br /><pre class="prettyprint"><br /><br />_debugged_app = None<br />app = ... build your wsgi app ...<br /><br />def main():<br /> # Only run the debugger in development.<br /> import os, sys<br /> if 'SERVER_SOFTWARE' in os.environ and os.environ['SERVER_SOFTWARE'].startswith('Dev'):<br /> # use our debug.utils with Jinja2 templates<br /> import debug.utils<br /> sys.modules['werkzeug.debug.utils'] = debug.utils<br /><br /> # don't use inspect.getsourcefile because the imp module is empty <br /> import inspect<br /> inspect.getsourcefile = inspect.getfile<br /> <br /> # wrap the application<br /> from werkzeug import DebuggedApplication<br /> global _debugged_app<br /> if _debugged_app is None:<br /> _debugged_app = app = DebuggedApplication(app, evalex=True)<br /> else:<br /> app = _debugged_app <br /></pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-64996360942238598142009-05-14T19:55:00.006+09:002009-05-14T20:29:27.296+09:00Google App Engine で Werkzeug のデバッガを使うWerkzeug という Python Web Framework があります。Werkzeug のデバッガはなかなかすぐれもので Web アプリでエラーが出るとインライン console が使えたりします。<br /><br />このデバッガを GAE の dev server で使う<a href="http://dev.pocoo.org/projects/werkzeug/wiki/UsingDebuggerWithAppEngine">方法</a>を作者の方が書いているのですが、この通りやるとインライン console が使えなかったりします。<br /><br />ちょっと調べてみたら意外と簡単に使えました。GAE の dev server だとリクエスト毎に DebuggedApplication のインスタンスが作り直されてしまうので、これを module global のシングルトンにしてやればオッケイでした。<br /><br /><pre class="prettyprint"><br /><br />_debugged_app = None<br />app = ... build your wsgi app ...<br /><br />def main():<br /> # Only run the debugger in development.<br /> import os, sys<br /> if 'SERVER_SOFTWARE' in os.environ and os.environ['SERVER_SOFTWARE'].startswith('Dev'):<br /> # use our debug.utils with Jinja2 templates<br /> import debug.utils<br /> sys.modules['werkzeug.debug.utils'] = debug.utils<br /><br /> # don't use inspect.getsourcefile because the imp module is empty <br /> import inspect<br /> inspect.getsourcefile = inspect.getfile<br /> <br /> # wrap the application<br /> from werkzeug import DebuggedApplication<br /> global _debugged_app<br /> if _debugged_app is None:<br /> _debugged_app = app = DebuggedApplication(app, evalex=True)<br /> else:<br /> app = _debugged_app<br /></pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-37935625419768692692009-03-20T17:42:00.007+09:002009-03-22T15:06:06.078+09:00Android hack-a-thon3/20に <a href="http://googlejapan.blogspot.com/2009/03/android-hackathon.html">Android hack-a-thon</a> に参加してきました。<br /><br />みなさん一日で色々作っています。すごいな〜<br /><br />私の方は、全くの初心者なので午前中は HelloWorld をやってみました。午後からは gdata のライブラリを使って Google Apps のグループアドレスを作成できるようなアプリケーションにトライ。<br /><br />gdata の jar をプロジェクトに取り込み、必要な画面を作るところまではうまくいったけど、 GroupService の認証をするところでうまくいかず、当日はここで断念。原因はSSL証明書の検証が失敗している事まではつきとめたんですけど...<br /><br />せっかくなので今日がんばって動かす事ができたので書きます。<br /><br />まず gdata の Java クライアントのソースを落としてきて、下記のパッチを当てます。内容はSSL証明書の検証が必ず成功するようにしているだけです。<br /><br /><pre class="prettyprint"><br />diff -uNr gdata/java/src/com/google/gdata/client/GoogleAuthTokenFactory.java gdata.mine/java/src/com/google/gdata/client/GoogleAuthTokenFactory.java<br />--- gdata/java/src/com/google/gdata/client/GoogleAuthTokenFactory.java 2009-02-12 05:09:34.000000000 +0900<br />+++ gdata.mine/java/src/com/google/gdata/client/GoogleAuthTokenFactory.java 2009-03-22 13:49:42.000000000 +0900<br />@@ -16,6 +16,13 @@<br /><br />package com.google.gdata.client;<br /><br />+import org.apache.http.conn.ssl.X509HostnameVerifier;<br />+import javax.security.cert.X509Certificate;<br />+import javax.net.ssl.HttpsURLConnection;<br />+import javax.net.ssl.SSLSession;<br />+import javax.net.ssl.SSLSocket;<br />+import javax.net.ssl.SSLException;<br />+<br />import com.google.gdata.util.common.base.CharEscapers;<br />import com.google.gdata.util.common.base.StringUtil;<br />import com.google.gdata.client.GoogleService.AccountDeletedException;<br />@@ -450,7 +457,27 @@<br /><br /> // Open connection<br /> HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();<br />-<br />+ if ( urlConnection instanceof HttpsURLConnection ) {<br />+ HttpsURLConnection ucs = (HttpsURLConnection) urlConnection;<br />+ ucs.setHostnameVerifier( new X509HostnameVerifier(){<br />+ @Override<br />+ public boolean verify(String arg0, SSLSession arg1) {<br />+ return true;<br />+ }<br />+ @Override<br />+ public void verify(String arg0, SSLSocket arg1) throws IOException {<br />+ }<br />+ @Override<br />+ public void verify(String arg0,<br />+ java.security.cert.X509Certificate arg1)<br />+ throws SSLException {<br />+ }<br />+ @Override<br />+ public void verify(String arg0, String[] arg1, String[]<br />+ arg2) throws SSLException {<br />+ }<br />+ });<br />+ }<br /> // Set properties of the connection<br /> urlConnection.setDoInput(true);<br /> urlConnection.setDoOutput(true);<br />diff -uNr gdata/java/src/com/google/gdata/client/appsforyourdomain/AppsGroupsService.java gdata.mine/java/src/com/google/gdata/client/appsforyourdomain/AppsGroupsService.java<br />--- gdata/java/src/com/google/gdata/client/appsforyourdomain/AppsGroupsService.java 2009-02-12 05:09:34.000000000 +0900<br />+++ gdata.mine/java/src/com/google/gdata/client/appsforyourdomain/AppsGroupsService.java 2009-03-22 14:10:47.000000000 +0900<br />@@ -37,7 +37,7 @@<br />public class AppsGroupsService extends AppsPropertyService {<br /><br /> public static final String BASE_URL =<br />- "http://apps-apis.google.com/a/feeds/group/2.0/";<br />+ "https://apps-apis.google.com/a/feeds/group/2.0/";<br /> public final String baseDomainUrl;<br /><br /> public static final String APPS_PROP_GROUP_ID = "groupId";<br />diff -uNr gdata/java/src/com/google/gdata/client/http/HttpGDataRequest.java gdata.mine/java/src/com/google/gdata/client/http/HttpGDataRequest.java<br />--- gdata/java/src/com/google/gdata/client/http/HttpGDataRequest.java 2009-02-12 05:09:35.000000000 +0900<br />+++ gdata.mine/java/src/com/google/gdata/client/http/HttpGDataRequest.java 2009-03-22 13:49:39.000000000 +0900<br />@@ -16,6 +16,13 @@<br /><br />package com.google.gdata.client.http;<br /><br />+import org.apache.http.conn.ssl.X509HostnameVerifier;<br />+import javax.security.cert.X509Certificate;<br />+import javax.net.ssl.HttpsURLConnection;<br />+import javax.net.ssl.SSLSession;<br />+import javax.net.ssl.SSLSocket;<br />+import javax.net.ssl.SSLException;<br />+<br />import com.google.gdata.util.common.xml.XmlWriter;<br />import com.google.gdata.client.AuthTokenFactory;<br />import com.google.gdata.client.GDataProtocol;<br />@@ -312,6 +319,29 @@<br /> }<br /> HttpURLConnection uc = (HttpURLConnection) requestUrl.openConnection();<br /><br />+ // Open connection<br />+ if ( uc instanceof HttpsURLConnection ) {<br />+ HttpsURLConnection ucs = (HttpsURLConnection) uc;<br />+ ucs.setHostnameVerifier( new X509HostnameVerifier(){<br />+ @Override<br />+ public boolean verify(String arg0, SSLSession arg1) {<br />+ return true;<br />+ }<br />+ @Override<br />+ public void verify(String arg0, SSLSocket arg1) throws IOException {<br />+ }<br />+ @Override<br />+ public void verify(String arg0,<br />+ java.security.cert.X509Certificate arg1)<br />+ throws SSLException {<br />+ }<br />+ @Override<br />+ public void verify(String arg0, String[] arg1, String[]<br />+ arg2) throws SSLException {<br />+ }<br />+ });<br />+ }<br />+<br /> // Should never cache GData requests/responses<br /> uc.setUseCaches(false);<br /><br /></pre><br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIOVBtj6B9nXkIvYMVDtF6Cclm9NinLBPdUrqizeq18HICXygs7eNa2MKwKxEoDt-CjcWivHGAJakN9Sw9TYXVFhjlAjuHcuOf2OTx963xvpOUbBQskLra4DhdDMX4mhCVARIQh7pIuIc/s1600-h/android-myfirstapp.png"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 320px; height: 266px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIOVBtj6B9nXkIvYMVDtF6Cclm9NinLBPdUrqizeq18HICXygs7eNa2MKwKxEoDt-CjcWivHGAJakN9Sw9TYXVFhjlAjuHcuOf2OTx963xvpOUbBQskLra4DhdDMX4mhCVARIQh7pIuIc/s320/android-myfirstapp.png" alt="" id="BLOGGER_PHOTO_ID_5315888316471000226" border="0" /></a><br /><br />あと gdata/java/build-src/core.xml のコンパイル箇所で android.jar に classpath を通せばコンパイルが通ります。(メッセージに従って mail.jar と activation.jar を用意する必要もあるけど)<br /><br />でき上がった gdata-appsforyourdomain-1.0.jar と gdata-core-1.0.jar を使うときちんと動かすことができました。手軽に Google Apps の Group を追加できるだけのしょーもないアプリですが...<br /><br /><br /><br />これを進化させて Google Apps の管理コンソールが作れそう。でも需要は無さそうだなorztmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com1tag:blogger.com,1999:blog-2144470051777636547.post-33185360787757385492009-03-05T08:09:00.004+09:002009-03-05T08:20:47.540+09:00Werkzeug の easteregg<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy_zy2d3oLfUC0n0cgxiEZQz5-HVeLaPErDbnoSgaAw-_8VF_VZdJs_iIOJB2NykyrMiZXOPzwK9MEFTz8t_u8QHmn6HcC-YuOSchO6D7kgSp6VDesVJ_ySXUIt55UB5WgqXB-n-ZGVx8/s1600-h/werkzeug-easteregg.png"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 211px; height: 282px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy_zy2d3oLfUC0n0cgxiEZQz5-HVeLaPErDbnoSgaAw-_8VF_VZdJs_iIOJB2NykyrMiZXOPzwK9MEFTz8t_u8QHmn6HcC-YuOSchO6D7kgSp6VDesVJ_ySXUIt55UB5WgqXB-n-ZGVx8/s320/werkzeug-easteregg.png" alt="" id="BLOGGER_PHOTO_ID_5309475921961167682" border="0" /></a><br /><br />僕も最近話題の Werkzeug を使ってみようと思いました。しかしまだ使うまでにいたらず、とりあえず easteregg を発見して喜んでいる状態です。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-63492358785169592832009-02-23T02:25:00.018+09:002009-02-26T13:05:44.013+09:00Gaebarを使ってみた<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU4dSMz6yPsHpLGqOJ0Jr6jnXANn53W6-O6Knb-KLXVAzeT_lf8RdhxMCkFoqESSjtRj0j3L7CjjViO57J1KDEnnpFr1Mp5td_UQ0Fcsp1v_KF5pKVZHwNBqbJcKa9BTc6_W0GLo_jdVM/s1600-h/gaebar.png"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 320px; height: 207px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU4dSMz6yPsHpLGqOJ0Jr6jnXANn53W6-O6Knb-KLXVAzeT_lf8RdhxMCkFoqESSjtRj0j3L7CjjViO57J1KDEnnpFr1Mp5td_UQ0Fcsp1v_KF5pKVZHwNBqbJcKa9BTc6_W0GLo_jdVM/s320/gaebar.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5306950300761101186" /></a><br />先日 <a href="http://app-gallery.appspot.com/">App Gallery</a> という GAE 上のサービスを始めてみました。サービスをやるからにはデータをバックアップしたいと思い、<a href="http://aralbalkan.com/1784">Gaebar</a> を試してみたのでご紹介します。<br /><br />なんと Gaebar の画面は django pony です!<br /><br />Gaebar は Google App Engine Backup And Restore の略で、文字通り Google App Engine 用のバックアップ・リストアツールです。ゲイバーと読むそうです。この名前が良いですねw<br /><br />Gaebar は Django のアプリケーションで簡単に使えます。GAE上で動く既存の Django アプリケーションにプラグインする事で、バックアップ・リストア機能を追加する事ができます。<br /><br />残念ながら今は Django で動いているアプリケーションでしか使えませんが、少し頑張れば同じ事は別のフレームワークでも可能だと思います。<br /><br /><span style="font-weight:bold;">準備する</span><br /><br />まずは Gaebar をダウンロードします。<br /><a href="http://github.com/aral/gaebar/tree/master">Gaebar github</a> から Download 機能を使ってアーカイブをダウンロードするか、下記のように git で clone しても良いです。<br /><pre class="prettyprint"><br />$ git clone git://github.com/aral/gaebar.git<br /></pre><br />またはあなたのプロジェクトが git を使っているなら下記のコマンドが良いかもしれません。<br /><pre class="prettyprint"><br />$ git submodule add git://github.com/aral/gaebar.git <br /></pre><br /><br />方法はともあれ、プロジェクト直下に gaebar というディレクトリができていれば大丈夫です。<br /><br />次に GAE の SDK にパッチを当てます。これは Gaebar がデータを python のファイルとしてダウンロードしてきてローカルのファイルシステムに保存するために必要なのです。方法が <a href="http://aralbalkan.com/1440">AralのBlog Entry</a> に乗っていますが、この情報は少し古いので、SDK-1.1.9では下記のパッチを使ってください。<br /><pre class="prettyprint"><br />--- ../google_appengine/google/appengine/tools/dev_appserver.py.org 2009-02-26 11:33:33.000000000 +0900<br />+++ ../google_appengine/google/appengine/tools/dev_appserver.py 2009-02-20 17:04:46.000000000 +0900<br />@@ -1274,6 +1274,7 @@<br /> allowed_symbols = self._WHITE_LIST_PARTIAL_MODULES[module.__name__]<br /> for symbol in set(module.__dict__) - set(allowed_symbols):<br /> if not (symbol.startswith('__') and symbol.endswith('__')):<br />+ module.__dict__['old_'+symbol] = module.__dict__[symbol]<br /> del module.__dict__[symbol]<br /> <br /> if module.__name__ in self._MODULE_OVERRIDES:<br /></pre><br /><br />注意) app-engine-patch を使う場合は、settings.py で DJANGO_STYLE_MODEL_KIND = False にしないと動きませんでした。(少なくとも私は動かせませんでした。)動かせた方がいたら教えてください :-)<br /><br />1. settings.py は下記のようにします。<br /><pre class="prettyprint"><br /> INSTALLED_APPS = (<br /> # Other apps...<br /> 'gaebar',<br /> )<br /></pre><br /><br />2. urls.py は下記のとおりです<br /><pre class="prettyprint"><br />urlpatterns = patterns('',<br /><br /> # ...other URLs<br /><br /> url(r'^gaebar/', include('gaebar.urls')),<br />)<br /></pre><br /><br />3. app.yaml に static folder を追加します。<br /><pre class="prettyprint"><br /># Static: Gaebar<br />- url: /gaebar/static<br /> static_dir: gaebar/static<br /></pre><br /><br />4. index.yaml にインデックスを追加します。一回 dev server で Gaebar を動かせば自動で追加されます。<br /><pre class="prettyprint"><br />- kind: GaebarCodeShard<br /> properties:<br /> - name: backup<br /> - name: created_at<br /></pre><br /><br />5. Gaebar の設定を settings.py に追加します。<br /><br />GAEBAR_LOCAL_URL: ローカル開発サーバの絶対 URL を書きます。<br />GAEBAR_SECRET_KEY: 秘密の文字列を記入します。誰にも秘密にしてください。<br />GAEBAR_SERVERS: dict でサーバーを列挙します。<br />GAEBAR_MODELS: tuple でバックアップするモデルを列挙します。<br /><br />App Gallery で使用している設定のサンプルを下記に掲載します。<br /><pre class="prettyprint"><br />GAEBAR_LOCAL_URL = 'http://localhost:8000'<br />GAEBAR_SECRET_KEY = 'replace_with_your_secret_key'<br /><br />GAEBAR_SERVERS = {<br /> u'Deployment': u'http://app-gallery.appspot.com',<br /> u'Staging': u'http://app-gallery-staging.appspot.com',<br /> u'Local Test': u'http://localhost:8080',<br /> }<br /><br />GAEBAR_MODELS = (<br /> (<br /> 'appgallery.models',<br /> (u'Application', u'ThumbnailImage'),<br /> ),<br /> (<br /> 'tags.models',<br /> (u'Tag', u'TagApplication')<br /> ),<br /> (<br /> 'favorites.models',<br /> (u'Favorite',)<br /> ),<br /> (<br /> 'ragendja.auth.custom_models',<br /> (u'User',),<br /> ),<br />)<br /></pre><br /><br />あと注意点がもう一つ。ローカル開発サーバーで django.contrib.csrf.middleware.CsrfMiddleware を使っているとバックアップがうまく行きませんでした。なので私はローカル開発サーバーでだけ CsrfMiddleware をオフにしています。<br /><br />やった!これで Gaebar が使用できます。使う前に、Gaebar がどんな風に動くか説明しておきますね。<br /><br />Gaebar は datastore のデータを Python code としてバックアップします。その Python code を実行する事でリストアします。バックアップは長くかかりますが、GAE はそれを許さないので、Gaebar は バックアップとリストアのプロセスを細切れにして ajax を使って繰り返し呼び出します。<br /><br />デフォルトでは Gaebar は 5 つの Entity をバックアップし Python code に変換し、1MB の制限を回避するために 300KB くらいのコード断片として datastore に保存します。バックアップの過程では、このすぐ後に、コード断片をローカルサーバー側にダウンロードして保存する事になります。<br /><br />もしあなたがもっと高い Quota を持っているなら、views.py をいじる事でこれらのデフォルト値を変えられます。<br /><br />Gaebar を使う時は、十分にテストしてから使ってくださいね。万が一データが消失しても、私や Aral は責任を持てませんので。<br /><br /><span style="font-weight:bold;">使い方</span><br /><br />A. リモートバックアップを取るには<br /><br />1. Gaebar をあなたのアプリケーションと一緒に GAE にデプロイします<br /><br />注意: デプロイする時に、gaebar/backups フォルダ内のバックアップデータも一緒にアップロードされる事を頭にいれておいてください。不必要ならバックアップデータは別の場所に移動した状態でデプロイすると良いでしょう。<br /><br />2. 本番サーバで Gaebar にアクセスします。(例: http://myapp.appspot.com/gaebar/ )<br /><br />3. Create New Backup ボタンをクリックします。<br /><br />* バックアップをする時に、ローカル開発サーバーが動作している必要があります。<br /><br />注意: reference error が発生した場合(存在しない Entity への ReferenceProperty)は、バックアップ時には無視されます。リストアすれば reference error は消滅します。<br /><br />注意: 今までバックアップした一番大きなデータストアは 18995 行で 223 code 断片、35MB ありました。ローカル開発サーバーへのリストアは一晩中かかりました(ローカル開発サーバーの datastore は激遅なので)。もし記録を破ったら aral@aralbalkan.com に教えてあげてください :-)<br /><br />注意2: バックアップ中の例外を詳しく見たい場合は、本番サーバーの settings.py で DEBUG_PROPAGATE_EXCEPTIONS = True と設定してください。<br /><br /><br />B. ローカルでリストアする<br /><br />開発中に、リアル世界のデータを使ってテストするために、ローカルの開発サーバーにデータをリストアできます。<br /><br />1. ローカル開発サーバの Gaebar ページに行きます。(例: http://localhost:8000/gaebar/ )<br />2. リストからリストアしたいバックアップデータを選びます。<br />3. 終わるまで待ちます。<br /><br />tips: あらかじめ /var/tmp/ にある datastore の history ファイルのバックアップを取った方が良いです。<br /><br />ローカルのデータストアは非常に遅いので、覚悟してください。<br /><br />C. ステージング環境にリストアする<br /><br />GAE にはバージョニング機能がありますが、datastore は別バージョン間でも共通です。ですからステージング用の別アプリを使用するのは良い考えでしょう。<br /><br />本番とは別のアプリケーションスロットを用意して、そこにリストアしたいバックアップデータと一緒に Gaebar をデプロイします。後はローカル開発サーバーでのリストアと同じです。<br /><br />D. 本番環境にリストアする<br /><br />本番環境でトラブルがあった場合などは、バックアップからリストアできます。データストアのデータは上書きされるので注意してください。<br /><br /><br />実際に App Gallery でもバックアップ・リストアしてみましたが、完璧に動いています。Gaebar はかなり便利なのでみなさんも使ってみてはいかがでしょうか。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com1tag:blogger.com,1999:blog-2144470051777636547.post-67195691970050723912009-02-11T03:15:00.009+09:002009-02-11T03:41:57.374+09:00Mercurial deps extensionMercurial で svn:externals みたいな事できないのか #mercurial irc で聞いてみたところ、<a href="http://www.selenic.com/mercurial/wiki/index.cgi/DepsExtension">deps extension</a>を使うと良いらしい。<br /><br />Mercurial に同梱されてないので別途<a href="http://hg.hacks-galore.org/aleix/hgdeps">このへん</a>から落として来て使う。<br /><br /><h2>設定</h2><br />deps.py を適当なところに置いて、hgrc に下記のように設定する。<br /><pre class="prettyprint">[extensions]<br />hgext.deps =<br /># 又は deps.py を hgext dir 以外に置いたなら下記のようにする<br /># deps = /path/to/deps.py<br /></pre><br /><br /><h2>ロケーション設定</h2><br />依存関係のロケーションを設定する。これも hgrc に書く。ここで紹介する例は libfoo が外部 Mercurial リポジトリにあって、libbar は外部の CVS で管理されている場合。<br /><br /><pre class="prettyprint">[deps]<br />aliases = libfoo, libbar<br />alias.libfoo = /path/to/libfoo<br />alias.libbar = :pserver:anonymous@cvs.server.org:/sources/bar<br />alias.libbar.command = cvs -z3 -d$source co -r $rev -d $dest bar<br /></pre><br /><br /><h2>実際の依存関係を設定</h2><br />これはコマンドで行う。<br /><pre class="prettyprint">$ hg deps -a libfoo -r 3a9b061bada1 -d lib/foo 0.9.1<br />$ hg deps -a libbar -r v0r7_8 -d lib/bar 0.9.1<br /><br />$ hg deps -a libfoo -r f24139319bdb -d lib/foo 1.0<br />$ hg deps -a libbar -r v0r8_0 -d lib/bar 1.0<br /></pre><br /><br />書式はこう<br /><br /><pre class="prettyprint">hg deps -a <ロケーション設定のalias名> -r <外部のリビジョン> -d <work dir内での場所> <本体のバージョン><br /></pre><br /><br />本体のバージョンに応じて外部リポジトリのリビジョンを変えたりできるわけか。<br /><br />で、これをすると下記のような .hgdeps というファイルが出来る。これも hg add するのが良いみたい。<br /><pre class="prettyprint">[1.0]<br />f24139319bdb libfoo lib/foo<br />v0r8_0 libbar lib/bar<br /><br />[0.9.1]<br />3a9b061bada1 libfoo lib/foo<br />v0r7_8 libbar lib/bar<br /></pre><br /><br /><h2>依存関係を持ってくる</h2><br /><br />下記のようにして clone するくらいしか機能は無いみたい。<br /><pre class="prettyprint">$ hg depsclone 1.0<br /></pre><br /><br />この例では、バージョン 1.0 用の外部依存関係を取ってくる事ができる。<br /><br />まあすごく便利っていうわけでも無いし、使いたい人も限られてるだろうけど、Mercurial で svn:externals みたいな事がしたい人は試してみてください。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-91484476904238705762009-01-31T10:43:00.003+09:002009-01-31T10:53:15.412+09:00Google Apps に Group 機能が来ましたいつの間にか Google Apps の有料版にグループ機能が来ました。もしかするとコントロールパネルを拡張版にしないと出てこないかもしれません。おそらく今までの maillist 機能を拡張した機能で、ざっと触ってみたところ<br /><br /><ol><br /><li>いくらでも入れ子に出来る</li><br /><li>メンバーのロールが Owner と Member の二種類ある</li><br /><li>ドメイン外のユーザを Owner としても Member としても追加できる</li><br /><li>このグループへの送信者を下記のグループ単位で制限できる</li><br /><ul><br /><li>Owners<br /><li>Members<br /><li>ドメイン内のユーザ<br /><li>Any mail address<br /></ul><br /><li>どうやら Google Document や Google Sites の invite の時に使えそう<br /></ol><br /><br />今までは Google Apps を導入するお客さんには別立てでメーリングリストのサーバ(mailmanとか)を勧める事が多かったけど、これからはあまり必要なくなるかもしれませんね。tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-64257535182495084142009-01-31T09:42:00.009+09:002009-01-31T10:33:06.794+09:00Python旅館第一回最近やる気が無かったのですが Python旅館に来たら嘘のようにやる気が復活しました。<br /><br />本当は原稿を書く予定だったんですが、隣に座った <a href="http://twitter.com/tokibito">tokibito</a> が <a href="http://d.hatena.ne.jp/nullpobug/20090106/1231216470">IE6 の怖いコレ</a> について教えてくれました。<br /><br />おお!こないだ公開した <a href="http://www.orkut.com/Main#AppInfo.aspx?appUrl=http%3A%2F%2Ftriviads.appspot.com%2Fstatic%2Fmy-trivia.xml&objs=&sn=&ref=SR">My Trivia</a> にもこのセキュリティホールあるし。早速直さないと。この OpenSocial アプリ ではユーザ入力の Trivia をサーバ側から json で返して、OpenSocial Canvas 側で表示しているのですが、とりあえず表示の時に document.createTextNode() してるから大丈夫だろうと思っていました。<br /><br />しかしこういう時は < や > を下記のように処理してあげないといけないという事ですね。(この文脈では obj にユーザ入力文字が入ってます)<br /><br /><pre class="prettyprint"><br /> obj = obj.replace('<', r'\u003C')<br /> obj = obj.replace('>', r'\u003E')<br /></pre><br /><br />しかしこんな簡単な修正に 1h くらいかかってしまって情けない...tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-59021190843013197152009-01-06T10:49:00.003+09:002009-01-06T10:55:35.053+09:00Blogger で Syntax Highlighting明けましておめでとうございます。今年はすっかり寝正月でした。<br /><br />突然ですが Blogger で手軽に Syntax Highlighting を使えたので書いてみます。<br />テンプレートのヘッダに下記を追加して<br /><pre class="prettyprint"><br /><link href='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css' rel='stylesheet' type='text/css'/><br /><script src='http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js' type='text/javascript'/><br /></pre><br /><br />body の onload ハンドラを設定します。<br /><pre class="prettyprint"><br /><body onload='prettyPrint()'><br /></pre><br /><br />実際のコード部には class 指定した pre タグを使います。<br /><pre class="prettyprint"><br /><pre class="prettyprint"><br /></pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com3tag:blogger.com,1999:blog-2144470051777636547.post-17270027290904956942008-10-11T11:17:00.006+09:002009-10-20T18:31:57.002+09:00GeoPt Property 用のカスタムウィジェットこないだ Maps API hack-a-thon Tokyo に参加してきました。その時作ったのが Google App Engine の GeoPt Property をとても扱いやすくするための<a href="http://paste.shehas.net/show/20/">ユーティリティクラス</a>です。<br />
<br />
この記事では、そのクラスを使って geopoint データを datastore に保存するサンプルコードをお見せします。始めるまえに、こちらの<a href="http://code.google.com/appengine/articles/djangoforms.html">djangoforms に関する記事</a>を読んでおくとコードがすぐ理解できるでしょう。<br />
<br />
まずはこの記事用のディレクトリを作ります。そして拙作の <a href="http://paste.shehas.net/show/20/">my_geopt.py</a> をその中にコピーし、さらに新しく3つのファイルを作ります。app.yaml, main.py と index.html です。<br />
<br />
ファイルを作成したら、開発用サーバーを機動して localhost にアクセスします。すると Google Map をクリックするだけで特定の GeoPt が指定できる事がわかります。<br />
<br />
また <a href="http://tm-test.appspot.com/">ライブデモ</a>もあります(少し複雑になってますが、基本は同じです)。<br />
<br />
Happy coding :-)<br />
<br />
app.yaml<br />
<pre class="prettyprint">application: geotest
version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
script: main.py
</pre><br />
<br />
main.py<br />
<pre class="prettyprint">from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
import os
import wsgiref.handlers
from google.appengine.ext import db
from google.appengine.ext.webapp import template
from google.appengine.ext.db import djangoforms
import my_geopt
class Place(db.Model):
name = db.StringProperty()
description = db.StringProperty()
geopoint = db.GeoPtProperty()
date = db.DateTimeProperty(auto_now=True)
class PlaceForm(djangoforms.ModelForm):
class Meta:
model = Place
class MainPage(webapp.RequestHandler):
def post(self):
data = PlaceForm(data=self.request.POST)
if data.is_valid():
# Save the data, and redirect to the view page
entity = data.save(commit=False)
entity.put()
self.redirect('/')
else:
# Reprint the form
contents = {'form': data}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, contents))
def get(self):
contents = {'form': PlaceForm()}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, contents))
application = webapp.WSGIApplication(
[('/', MainPage),],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
</pre><br />
<br />
index.html (Please replace 2 YOURAPIKEYs with your Maps API Key)<br />
<pre class="prettyprint"><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>GeoPtWidgets Example</title>
<script src="http://maps.google.com/maps?file=api&v=2&key=YOURAPIKEY"
type="text/javascript"></script>
<script src="http://www.google.com/uds/api?file=uds.js&v=1.0&key=YOURAPIKEY"
type="text/javascript"></script>
<script src="http://www.google.com/uds/solutions/localsearch/gmlocalsearch.js"
type="text/javascript"></script>
<style type="text/css">
@import url("http://www.google.com/uds/css/gsearch.css");
@import url("http://www.google.com/uds/solutions/localsearch/gmlocalsearch.css");
</style>
</head>
<body onunload="GUnload()">
<h1>GeoPtWidgets Example</h1>
<form method="post" action="/">
{{ form.as_p }}
<input type="submit" value="register"/>
</form>
</body> </html>
</pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com0tag:blogger.com,1999:blog-2144470051777636547.post-57509225810484480672008-10-11T09:59:00.016+09:002009-10-20T18:30:27.173+09:00A custom widgets for GeoPt PropertyI attended a Maps API hack-a-thon in Tokyo. There I wrote some <a href="http://paste.shehas.net/show/20/">utility class</a> which helps us to handle GeoPt Property very easy.<br />
<br />
In this article, I'll show you some sample code which store geopoint data by just clicking on Google Map. Before we go, perhaps you can read the <a href="http://code.google.com/appengine/articles/djangoforms.html">djangoforms documentation</a>. This will help you to understand the codes in this article. <br />
<br />
Please create a directory for this article, copy <a href="http://paste.shehas.net/show/20/">my_geopt.py</a> to the directory and create 3 brand new files app.yaml, main.py, and index.html. That's all.<br />
<br />
After creating these files, start local dev server, get access to localhost, and see that you can specify a particular GeoPt by just clicking on the Google Map.<br />
<br />
And <a href="http://tm-test.appspot.com/">here is a live demo</a>(slightly more complicated, but basically almost the same).<br />
<br />
Happy coding :-)<br />
<br />
app.yaml<br />
<pre class="prettyprint">application: geotest
version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
script: main.py
</pre><br />
<br />
main.py<br />
<pre class="prettyprint">from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
import os
import wsgiref.handlers
from google.appengine.ext import db
from google.appengine.ext.webapp import template
from google.appengine.ext.db import djangoforms
import my_geopt
class Place(db.Model):
name = db.StringProperty()
description = db.StringProperty()
geopoint = db.GeoPtProperty()
date = db.DateTimeProperty(auto_now=True)
class PlaceForm(djangoforms.ModelForm):
class Meta:
model = Place
class MainPage(webapp.RequestHandler):
def post(self):
data = PlaceForm(data=self.request.POST)
if data.is_valid():
# Save the data, and redirect to the view page
entity = data.save(commit=False)
entity.put()
self.redirect('/')
else:
# Reprint the form
contents = {'form': data}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, contents))
def get(self):
contents = {'form': PlaceForm()}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, contents))
application = webapp.WSGIApplication(
[('/', MainPage),],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
</pre><br />
<br />
index.html (Please replace 2 YOURAPIKEYs with your Maps API Key)<br />
<pre class="prettyprint"><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>GeoPtWidgets Example</title>
<script src="http://maps.google.com/maps?file=api&v=2&key=YOURAPIKEY"
type="text/javascript"></script>
<script src="http://www.google.com/uds/api?file=uds.js&v=1.0&key=YOURAPIKEY"
type="text/javascript"></script>
<script src="http://www.google.com/uds/solutions/localsearch/gmlocalsearch.js"
type="text/javascript"></script>
<style type="text/css">
@import url("http://www.google.com/uds/css/gsearch.css");
@import url("http://www.google.com/uds/solutions/localsearch/gmlocalsearch.css");
</style>
</head>
<body onunload="GUnload()">
<h1>GeoPtWidgets Example</h1>
<form method="post" action="/">
{{ form.as_p }}
<input type="submit" value="register"/>
</form>
</body> </html>
</pre>tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com1tag:blogger.com,1999:blog-2144470051777636547.post-78166616147667618982008-07-13T23:41:00.022+09:002008-07-14T08:11:58.921+09:00Using the newest zipped pytz on GAEI posted <a href="http://takashi-matsuo.blogspot.com/2008/07/using-zipped-pytz-on-gae.html">an entry about how to use zipped pytz on GAE</a>[1].<br /><br />1. <a href="http://takashi-matsuo.blogspot.com/2008/07/using-zipped-pytz-on-gae.html">http://takashi-matsuo.blogspot.com/2008/07/using-zipped-pytz-on-gae.html</a><br /><br />It works well with older versions of pytz. Stefano pointed out that my method doesn't work with the newest pytz distribution. This article describes how to use the newest zipped pytz on GAE.<br /><br />First, retrieve the <a href="http://pypi.python.org/pypi/pytz/">newest pytz from pypi</a>[2] and extract the archive.<br /><br />2. <a href="http://pypi.python.org/pypi/pytz/">http://pypi.python.org/pypi/pytz/</a><br /><code><br />$ tar xjf pytz-2008c.tar.bz2<br />$ cd pytz-2008c/pytz<br />$ zip -q zoneinfo.zip `find zoneinfo -type f ! -name '*.pyc' -print`<br />$ rm -rf zoneinfo<br /></code><br />After that, you have to edit pytz/__init__.py and modify open_resource function to use zipped zoneinfo database like following:<br />(pkg_resources stuff were struck out. Thank you again Stefano!)<br /><pre><code><br />def open_resource(name):<br />"""Open a resource from the zoneinfo subdir for reading.<br /><br /><strike>Uses the pkg_resources module if available.</strike><br />"""<br /> import zipfile<br /> from cStringIO import StringIO<br /><br /><strike>if resource_stream is not None:<br /> return resource_stream(__name__, 'zoneinfo/' + name)<br />else:</strike><br /> name_parts = name.lstrip('/').split('/')<br /> for part in name_parts:<br /> if part == os.path.pardir or os.path.sep in part:<br /> raise ValueError('Bad path segment: %r' % part)<br /> zoneinfo = zipfile.ZipFile(os.path.join(os.path.dirname(__file__),<br /> 'zoneinfo.zip'))<br /> return StringIO(zoneinfo.read(os.path.join('zoneinfo', *name_parts)))<br /></code></pre><br />Then, the only thing to do is to copying your original pytz directory into your application directory.<br /><code><br />$ cd ../<br />$ cp -r pytz /your/application/directory<br /></code><br /><br />If you'd like to avoid using CPU to unzip zoneinfo data every time, perhaps you could use following function:<br /><pre><code><br />from google.appengine.api import memcache<br />import logging<br />import pytz<br />from pytz import timezone<br /><br />def getTimezone(tzname):<br /> try:<br /> tz = memcache.get("tz:%s" % tzname)<br /> except:<br /> tz = None<br /> logging.debug("timezone get failed: %s" % tzname)<br /> if tz is None:<br /> tz = timezone(tzname)<br /> memcache.add("tz:%s" % tzname, tz, 86400)<br /> logging.debug("timezone memcache added: %s" % tzname)<br /> else:<br /> logging.debug("timezone memcache hit: %s" % tzname)<br /><br /> return tz<br /></code></pre><br /><br />Happy coding :-)tmatsuohttp://www.blogger.com/profile/09975209669204600556noreply@blogger.com11