JournalDirectory

I find it useful to map out the URL design for a set of related objects and then start to work on the UI. In part five we did this for EntryUI. Lets get to work on the UI 'container' for our Entries, JournalDirectory. In this first cut we'll implement enough to display a list of recent entries and put stubs in place for Atom and RSS feeds.

class JournalDirectory(Directory):
    """
    This class provides the functionality to display recent entries,
    to navigate to a specific entry, and to present RSS and Atom feeds.

    Our application urls for this UI component will be:

    ../             "index" or recent entries
    ../1234/        calls the EntryUI "index" method
    ../new          create a new Entry
    ../index.rss
    ../index.xml
    """

    def get_exports(self):
        yield ('', 'index', None, None)
        yield ('index.rss', 'rss', 'RSS', 'RSS feed for this journal')
        yield ('index.xml', 'atom', 'Atom', 'Atom feed for this journal')

    def __init__(self, journal):
        require(journal, Journal)
        self.journal = journal

    def index [html] (self):
        title = "%s's journal" % self.journal.get_name()
        header(title)
        '<p>This is the journal of '
        self.journal.get_name()
        '</p>'
        for e in self.journal.get_recent_entries(count=10):
            '<p>%s %s %s</p>' % (e.key, e.get_text().get_format(),
                                 href(str(e.key),e.get_title()))
        footer(title)

    def new [html] (self):
        ' not implemented '

    atom = rss = new

Looking at the code above it you can see that we've provided no way of navigating to individual entries. In index we've generated links which look like this:

<a href="./1234">Some title</a>

What we've not done is provide a way of resolving that part of the URL path. Lets say for example we have a site:

http://mikewatkins.ca/blog

With entries:

http://mikewatkins.ca/blog/1234
http://mikewatkins.ca/blog/3456

How does the QP application resolve these paths?

_q_lookup

Unlike some web application frameworks which use regular expressions to map URL namespaces to objects/methods or functions, QP uses the concept of object publishing via object traversal. We've seen how a UI object that exposes methods can be called; _q_lookup provides us the ability to return "objects". Lets write a _q_lookup method for JournalDirectory that returns an EntryUI object for a specific journal Entry.

def _q_lookup(self, component):
    try:
        key = int(component)
    except ValueError:
        return not_found('%r is not a valid journal entry identifier.' % component)
    try:
        entry = self.journal.get_entry(key)
    except KeyError:
        return not_found('%r was not found.' % key)
    return EntryUI(entry)

At this point we have working JournalDirectory and EntryUI objects. Lets now replace our temporary demonstration code in the applications master controller with a connection to our Durus database and real data.

As you might imagine, with this object / UI pattern it would be easy to add support for multiple journals. Objects and their corresponding UI might look like:

Object              UI
--------------------------------------------
Entry               EntryUI
Journal             JournalDirectory
JournalDatabase     JournalDatabaseDirectory

Wiring up the application

For simplicities sake, we are going to first wire up our JournalDirectory to SiteDirectory which we last talked about in conjunction with the site's primary "driver", slash.qpy, last discussed in part five.

class SiteDirectory(Directory):

    def get_exports(self):
        yield ('', 'index', 'Home', 'The Home page of this site.')
        yield ('blog', 'blog', 'Weblog', None)

    def index [html] (self):
        title = get_site().get_name()
        header(title)
        '<p><strong>It worked!</strong></p>'
        '<p>The <strong>%s</strong> application lives at <br /><tt>' % title
        get_site().get_package_directory()
        '</tt></p>'
        footer()

    mw_journal = get_publisher().get_root()['mw_journal']
    blog = JournalDirectory(mw_journal)

Or, we could wire up JournalDirectory as the 'root' of our application like this:

class SiteDirectory(JournalDirectory):

    def __init__(self):
        # I've implemented the JournalDatabase object and migrated some
        # data into a Journal.
        mw_journal = get_publisher().get_root()['journals'].get('mw')
        JournalDirectory.__init__(self, mw_journal)

Going with the latter example, at this point we can restart the qp application (qp blog restart) and visit http://localhost:8011/ and we should see a list of available posts displayed which we can navigate to. Clearly we need to do a bunch more work - implement a site look and feel; convert RST, Markdown, and Textile formatted posts into HTML, implement RSS and Atom feeds, but the basics of a journal or weblog application have come together. Fleshing out the display of existing data will be our next step.

We still can't create, edit, or delete journal entries - we'll discuss that in a soon to be forthcoming installment in this series.

Durus, the database you already know

Now I know what you are thinking. I think. Well, that is my theory and it is mine and I own that theory. My theory is that you are thinking:

"Object database? what sort of weird and strange alchemy is that? Fear the unknown! Down with the unknown! Destroy the unknown with DELETE FROM queries!" -- you

While object databases are not exactly in commonplace use by the IT industry, within the Python community, there is a long history of kinship with object databases with ZODB, the Zope Object Database, arguably being the most well known example.

Durus is patterned after ZODB, and indeed was written by developers who had used ZODB extensively. Visit the Durus pages for more information on their rationale for reinventing this particular wheel; from my own experience I can only say that Durus is small and easy to read and understand.

What exactly is an object database? Put simply, Durus and ZODB allow you to persist your Python objects. Its more than pickle but not unlike pickle in some respects.

% qp -i blog
Profile, connection, publisher, root, sessions, site, users
->>

->> pu

->> from durus.persistent_dict import PersistentDict
->> mydict = PersistentDict()
->> root['test'] = mydict
->> connection.commit()
->>
%

% qp -i blog
Profile, connection, publisher, root, sessions, site, test, users
->>

->> test
<PersistentDict 17020>
->> test.items()
[]

->> test[1] = 'My first persistent data'
->> connection.commit()
->>

% qp -i blog
Profile, connection, journals, publisher, root, sessions, site, test, users
->> test.items()
[(1, 'My first persistent data')]
->>

Durus Mini FAQ

What about performance?

This is too difficult a question to answer simply, but its been my experience that I have been able to use Durus, instead of a SQL database (Postgres is my personal favorite among the open source databases), far more often than not. You won't put an on-line banking system processing millions of transactions a day on to Durus or ZODB; but you might base on Durus a complex company inventory system, even if there are hundreds of thousands of items and related history. Third party solutions marry Durus with relational databases as a back-end to Durus (transparent to the application) to extend Durus (ZODB has similar approaches I'm told) even further.

What about SQL / queries? How will I ever live?

One of the challenging things for a SQL-oriented developer (that was me, some time ago) is to start thinking in pure-Python again. Its not hard, but it does take some realignment of thought before it comes naturally - at least for me. Being able to dispense with relational thinking in the SQL sense brings a lot of design freedom.

What about sharing data with other systems?

My approach has been to export data as CSV or DIF for import into other systems SQL databases, or to provide APIs such as XML-RPC or REST / JSON approaches for other applications themselves, or to use RSS or Atom feeds when it makes sense.

The bottom line: Durus objects are Python objects. You've already invested in learning and knowing Python, so you already know Durus, so there is no time-to-learn downside to spending some time with Durus now. Lets press on.

Entries with no home

In part three of this series we turned a simple Entry object into a full partner of a Durus database merely by subclassing PersistentObject instead of the standard Python new-style class object. In part four we kicked things up a notch by fleshing out our Entry object with specifications provided by the QP module qp.lib.spec.

What we have not done, yet, is provide a place for our journal entries to 'live'. We need a container for Entry, and early on we decided to call that container Journal. We are really going to kick things up a notch by levering off of functionality provided by QP in qp.lib.keep. A Keep is a mapping of Keyed items using an integer as a key. Lets enhance Entry first, then we'll write some unit tests for Journal, and then write Journal itself.

All the code for the end-result objects will be available at the conclusion of this series, but for you folks following along at home, lets dive in and re-edit our journal.py and clean up our Entry object first. For brevity's sake I have included imports relevant to both Entry and the Journal object we will be writing.

from dulcinea.base import DulcineaPersistent
from dulcinea.sort import attr_sort
from qp.lib.keep import Keep, Keyed, Stamped
from qp.lib.spec import add_getters_and_setters, boolean, both, datetime_with_tz
from qp.lib.spec import init, pattern, string, spec
from qp.pub.user import User


class Entry(DulcineaPersistent, Keyed, Stamped):
    """
    An entry in a journal.
    """
    title_is = spec(
        (string, None),
        "A string briefly describing the Entry")
    text_is = spec(
        (string, None),
        "The entry conten")
    published_is = spec(
        boolean,
        "Boolean indicating if Entry can be published")
    author_is = spec(
        User,
        "User responsible for creating entry")
    created_is = datetime_with_tz

    def __init__(self, author):
        Keyed.__init__(self)
        Stamped.__init__(self)
        init(self, author=author, created=self.stamp, published=False)

add_getters_and_setters(Entry)

Lets now write Journal but before we write it, lets write the tests we want it to pass, first, and then write the object. Typically you might write only some of these tests, at least until you become familiar with the various features of the QP and Dulcinea libraries. In our ./test/utest_journal.py we'll add another test.

from parlez.journal import Journal

class JournalTest(UTest):
    # we'll write this first, and then write Journal

    def _pre(self):
        # set up a journal which we'll use for most tests.
        self.j = Journal('science', User('einstein'))
        # it is automatically taken down following each individual test

    def init_test(self):
        # we want Journal to have a URL name and an owner, so force it
        Journal('musings', User('joe'))

    def create_entry_test(self):
        assert isinstance(self.j.create_entry(), Entry)

    def add_test(self):
        e = self.j.create_entry()
        self.j.add(e)
        assert e in self.j.get_all_entries()
        assert e == self.j.get_entry(1)

    def only_published_test(self):
        # nothing in
        assert self.j.get_all_entries() == []
        e = self.j.create_entry()
        self.j.add(e)
        e_published = self.j.create_entry()
        e_published.set_published(True)
        self.j.add(e_published)
        assert e not in self.j.get_entries()
        assert e_published in self.j.get_entries()
        # publish e now
        e.set_published(True)
        assert e in self.j.get_entries()
        # both should be in reverse sorted result, e last
        assert [e_published, e] == self.j.get_recent_entries()

if __name__ == '__main__':
    EntryTest()
    JournalTest()

I've kept this briefer than I'd like it to be, as there are some other tests we need to write to completely cover our Journal object, but these tests of primary functionality - add, retrieve, retrieve all and sort - should give you the spirit of what we are trying to achieve here.

PyBlosxom to Journal Conversion

A common challenge: you've got data in one system and need to move it into a Durus database. A script to perform this task will be included in full at the end of this series. For now lets sketch out what we need to do, and look at how to access an application's Durus database from a script.

Pyblosxom maintains its files in a hierarchy that looks like something like this:

../entries/categoryname/file1.txt
../entries/categoryname/someotherfile.rst
../entries/python/2007-06-08-08-44.rst

And so on. My particular installation uses a plugin which parses the entry date from the file name if it is formatted as a datetime in the form of yyyy-mm-dd-hh-mm.ext, so for files formatted like that I can set Entry.created to a datetime parsed from the filename. Otherwise, I need to stat the file and get its creation date from the operating system, which isn't always reliable (in the case of edits and hapless administrators).

The file contents are simple for me to parse - content is either plain text, or in my instance, mostly Textile formatted with a sprinkling of reST and Markdown.:

Some article title
#author Mike Watkins
The article content.

.h2 A subtitle

More content. Etc.

I never used the #author directive; some files use the #parser directive to indicate which formatter should be used; most rely on file extensions (.rst, .txt, .mkd).

Ultimately my script needs to deliver to me:

• Entry date
• Format
• Title
• Content

And, if I intend to preserve the URLs (am debating this now... I really dislike the existing bloxsom / Pyblosxom URL design) I'll need to carry that information forward too. For now, lets assume we have a mapping containing file paths as keys and a list with the four above noted data elements to work with, and write a script to import that information into Durus.

from qp.lib.site import Site
from parlez.journal import Entry, Journal

def bloxsom_to_mapping(entrypath):
    # here you'll deal with the specifics - see a future article
    data = {}
    # ...
    return data

def add_journal_entries(data, journal):
    for path, entry_data in data:
        # path I might store, or some component of it, in the Entry
        # object to facilitate mapping old to new URLs in the future.
        # for now, just ignoring it
        created, format, title, content = entry_data
        entry = journal.create_entry()
        entry.set_format(format)
        entry.set_title(title)
        entry.set_text(content)
        # normally we don't bypass getters/setters
        entry.created = created
        entry.stamp = created
        journal.add_entry(entry)

if __name__ == '__main__':
    BLOXSOM_ENTRY_PATH = '/home/mw/bloxsom/entries'
    APP_NAME = 'blog'
    JOURNAL_NAME = 'mw'
    USER_ID = 'mw'

    # the Site object gives us the ability to access
    # configuration information and live objects
    site = Site(APP_NAME)
    pub = site.get_publisher()
    root = pub.get_root()
    users = root['users']
    # make sure I exist in Users
    if USER_ID not in users:
       user = pub.create_user(USER_ID)
       users.add(user)
    if 'journal' not in root:
        journal = Journal(JOURNAL_NAME, user)
        root['journal'] = journal
    # move bloxsom data into Entry/Journal
    add_journal_entries(bloxsom_to_mapping(BLOXSOM_ENTRY_PATH),
                        journal)
    # made it here, commit everything to the database
    pub.get_connection().commit()
    # that's it!

Next Installment

When we return in part seven of this series we will further flesh out our UI objects for Entry and Journal, adding methods for creating and editing objects. At that point we'll have a basic journal or weblog application ready to deploy to the world. Subsequent articles will add more functionality.

QP Applications

In part two we created file system hierarchies for both a library and our project application. Since we plan on reusing our journal objects and related UI, those components will live in our library and our application itself will import them.

The fundamental requirements of any QP application are:

1. The application(s) must live somewhere in the QP search path. Issue qp -h at the command line for details.
2. Applications must define a module called either slash.py or slash.qpy. Both contain Python code, qpy files are QP template-aware Python modules which we'll talk about more shortly.
3. The slash module must expose two objects, a SitePublisher which defines the publishing environment and any application specific customizations, as well as a SiteDirectory which defines the "root" directory of the application.

We generated a basic application skeleton using a tool cooked up just for this article series, mkqpapp.py. This created in ~/qp_apps/blog the following hierarchy:

CHANGES
README
__init__.py
bin/
doc/
slash.qpy
test/
ui/
    test/
var/

Ignoring the obvious or already discussed, lets explore files of significance:

1. slash.qpy as discussed earlier can be thought of as the driver or root application directory of our new application.

2. __init__.py defines the directory as a package; however should there be any qpy modules in the given directory, __init__.py must include two lines of code:

   from qpy.compile import compile_qpy_files
   compile_qpy_files(__path__[0])
   

   QPY, which was installed in part one of this series, provides unicode and quote-safe Python templating, in a manner quite different than all other Python templating approaches. Diligent use of qpy can reduce or eliminate your application's risk of being exploited by cross site scripting attacks, and SQL injection attacks, if your application uses SQL.

   No magic or import hooks are required to integrate QPY with regular Python code; the two lines in __init__.py ensure that QPY files are compiled as regular Python code. Its a small price to pay for much flexibility as we'll see soon.

Exposing Objects, Methods or Functions on the Web

Lets now open the template slash.qpy module, explore, and make some changes. If you've not already started the template application created in part two, launch it now:

qp blog start

Visit the application at http://localhost:8011/ and you'll see:

It worked!

The blog application lives at /usr/home/yourhomedir/qp_sites/blog.

Lets edit slash.qpy and make some changes. First of all, since we've not discussed https secure http communications at all, yet, lets comment out some of the configuration:

class SitePublisher (DurusPublisher):

    configuration = dict(
        durus_address=('localhost', 7011),
        http_address=('', 8011),
        # as_https_address=('localhost', 9011),
        # https_address=('', 10011),
        )

The QP README contains a recipe for using stunnel for providing https SSL abilities to your QP applications.

class SiteDirectory(Directory):

    def get_exports(self):
        yield ('', 'index', 'Home', 'The Home page of this site.')

    def index [html] (self):
        title = get_site().get_name()
        header(title)
        '<p><strong>It worked!</strong></p>'
        '<p>The <strong>%s</strong> application lives at <br /><tt>' % title
        get_site().get_package_directory()
        '</tt></p>'
        footer()

def hello [html] (self):
    '<p>Hello, world</p>'

qp blog restart

qp -l blog

2007-06-06 19:45:04 404 .007931 127.0.0.1 - - 3005 http GET /hello - Mozilla/5.0_

def get_exports(self):
    yield ('', 'index', 'Home', 'The Home page of this site.')
    yield ('hello', 'hello', 'Greetings', 'A page for salutations')

form = dict(
    title='User supplied title',
    text='Malicious user supplied <script>destroy_you_all()</script>')

def hello [html] (self):
    '<p>Hello, world, I'm a malicious user and posted this:</p>'
    '<h2>%(title)s</h2>' % form
    '<p>%(text)s</p>' % form

Hello, world, I am a malicious user and posted this:

User supplied title

Malicious user supplied <script>destroy_you_all()</script>

Entry UI

Our last task for this installment is to flesh out the basic UI for the Entry object:

from qp.fill.directory import Directory
from qp.lib.spec import require
from qp.pub.common import header, footer
from parlez.journal import Entry, Journal, JournalDatabase

# this is for demonstration only.
from qp.pub.user import User
test_entry = Entry(User('foo'))
test_entry.set_title('An example journal entry')
test_entry.set_text("To QPY, or not to QPY, <that> is the question & what's for dinner?")

class EntryUI(Directory):
    """
    This class provides the functionality to display a single entry,
    edit a new or existing entry, delete an entry, and to represent
    an entry in different ways including RSS or Atom.

    Our application urls for this UI component will be:

        ../1234/
        ../1234/edit
        ../1234/delete
        ../1234/index.rss
        ../1234/index.xml
    """

    def __init__(self, component=test_entry):
        require(component, Entry)
        self.entry = component

    def get_exports(self):
        yield ('', 'index', None, None)
        yield ('edit', 'edit', 'Edit Entry', None)
        yield ('delete', 'delete', 'Delete Entry', None)
        yield ('index.rss', 'rss', 'RSS', 'RSS feed for this entry')
        yield ('index.xml', 'atom', 'Atom', 'Atom feed for this entry')

    def index [html] (self):
        # python docstrings can't be included in [html] templates
        # this template could also deliver the output of other
        # template engines such as Genshi, Cheetah, Mako, etc.
        # lets stick with QPY for now.
        title = self.entry.get_title()
        header(title)
        '<h1>%s</h1>' % title
        self.entry.get_text()
        footer(title)

    def atom [html] (self):
        'Not Implemented'

    # lets just point all these methods to atom / not implemented for now until
    # we implement each
    edit = delete = rss = atom

If you want to race ahead and see what this looks like from the browser side, update slash.qpy by adding an import, changing get_exports and SiteDirectory as follows (don't forget to restart the application afterwards):

# import your UI object from wherever you put it.
from parlez.ui.journal import EntryUI

Modify get_exports as such:

def get_exports(self):
    yield ('', 'index', 'Home', 'The Home page of this site.')
    yield ('hello', 'hello', 'Greetings', 'A page for salutations')
    yield ('test_entry', 'test_entry', 'EntryUI Test', None)

Modify SiteDirectory by adding an attribute which points to an instance of EntryUI:

class SiteDirectory(Directory):
    # snip

    test_entry = EntryUI()

And then restart the app (qp blog restart) and visit the following URL's:

• http://localhost:8011/test_entry (note what happens)
• http://localhost:8011/test_entry/
• http://localhost:8011/test_entry/edit
• http://localhost:8011/test_entry/delete
• http://localhost:8011/test_entry/index.rss
• http://localhost:8011/test_entry/index.xml

Next Installment

When we return in part six of this series we will inject some real Entry data into our Durus database and explore how we will work with Durus objects interactively, from our application, and from scripts.

After that we'll turn our application shell into a functioning application by stripping out the quick and dirty demonstration data hacks and completing our "controller" logic for EntryUI and Journal user interface classes.

Specifications: Contracts for Busy Developers

The QP package has a wonderful module, qp.lib.spec which deserves to get more attention whether QP does or not.

spec provides an easy way to declare or specify what various object attributes should contain, without littering your code with miles of asserts and other test. Examples will make things clear - lets take our too-simple Entry object and beef it up. First the original database-aware object:

from durus.persistent import PersistentObject

class Entry(PersistentObject):
    title = None
    text = None
    author = None
    created = None

Out of control: Clearly the object as its described presents a problem - its just an empty 'bag' with no constraints on what a user/developer might attempt to do with it. For example, whether intended or not, our dumb object allows for all sorts of questionable attribute assignments, as shown in this interactive session:

->> e = Entry()
->> e.title = 'my title'  # so far so good
->> e.title = None        # probably reasonable
->> e.title = 123.456     # not at all what we want
->> e.created = 3.1415
->> e.some_new_attribute = "foo"

The dynamic nature of Python is both blessing and curse at times; the above object requires lots of additional code to ensure that data it manages is what the developer intended, leading to significant code expansion via tests and assertions, reducing readability along the way.

In control: There is another way. Lets use spec and apply specifications and some helper methods:

from durus.persistent import PersistentObject
from qp.lib.spec import datetime_with_tz, spec, string
from qp.lib.spec import add_getters_and_setters

class Entry(PersistentObject):

    title_is = spec(
        (string, None),
        'A short description of the entry')
    text_is = spec(
        (string, None),
        'The full text of the entry')
    author_is = spec(
        User,
        'The individual responsible for the content of the entry')
    created_is = datetime_with_tz

add_getters_and_setters(Entry)

A spec can be a simple type assignment (see created_is above) or make use of the spec function which allows for a certain amount of self-documentation which can be very helpful. Arguments supplied in tuple imply the specification either, or you can spell it out: either(string, None).

Now lets run through the same interactive session as before:

->> e = Entry()
->> e.set_title('my title')
->> e.set_title(None)
->> e.set_title(123.456)
Traceback (most recent call last):
  File "<input>", line 2, in <module>
  File "/usr/local/lib/python2.5/site-packages/qp/lib/spec.py", line 725, in f
    require(value, getattr(klass, name + '_is'))
  File "/usr/local/lib/python2.5/site-packages/qp/lib/spec.py", line 171, in require
    raise TypeError(error)
TypeError:
  Expected: (string, None)
  A short description of the entry
  Got: 123.456

Aha, for the cost of a pair of get and set methods (no moaning please, we didn't even have to write the getter and setter ourselves and they don't clutter the code), we've effectively constrained what type of data can be assigned to the title attribute of Entry, while also preserving the easy to read nature of the original, simple, code.

To demonstrate the utility of spec, I've pulled a number of examples from Dulcinea and my own code. As you can see, there is great flexibility provided:

date_is = datetime

approvals_is = spec(
    sequence(DulcineaUser, set),
    "The users who agree the issue is resolved.")

issues_is = spec(
    mapping({string:Issue}, PersistentDict),
    "Mapping of issue IDs to issues.")

id_is = spec(
    pattern('^[-A-Za-z0-9_@.]*$'),
    "unique among users here")

# some specs are referred to over and over again

datetime_without_tz = both(datetime, with_attribute(tzinfo=None))
datetime_with_tz = both(datetime, with_attribute(tzinfo=no(None)))
email_pattern = pattern("^.+@.+\..{2,4}$")
existing_user = both(User, with_attribute(id=no(None)))
hex_pattern = pattern('[a-fA-F0-9]*$')

# reuse them

email_is = email_pattern
id_is = spec(
    hex_pattern,
    'a lower case alphanumeric pattern')

# use the specifications in tests (more powerful and cleaner than
# testing for type and instance alone)

require(thing, either(list, tuple))
match(a_user, existing_user) # returns boolean

My experience is that you can create fairly complex specifications that remain very readable. Have a complex object that has to be "just so" before it is committed to a database in a transaction? Specify everything, and check it for sanity with a one line assertion: assert get_spec_problems(theobject_instance) == [] and you are done.

Testing, Testing, One Two Three

As you might imagine, it becomes easier to write unit tests when our objects are so highly specified. Sancho, a unit testing framework also from the same development shop from which QP originates, is designed for projects and teams who prefer to leave code in a working state, all or most of the time.

Tests live in ./test, one level down from our objects being tested, and there is no __init__.py. A utility, urun.py, will execute one test supplied on the command line, or all tests in the test subdirectories in the current directory and below. Lets write one for Entry:

# /www/lib/parlez/test/utest_journal.py
from parlez.journal import Entry
from sancho.utest import UTest, raises

entry_text = '''This is a blog entry.\n\n*We hope you like it*.'''

class EntryTest(UTest):

    def init_test(self):
        Entry()

    def entry_test(self):
        joe = User('joe')
        e = Entry()
        e.set_author(joe)
        # a string causes a TypeError, authors must be User instances
        raises(TypeError, e.set_author, 'Joe')
        assert e.get_author() == joe
        assert e.get_created() == e.get_stamp()
        e.set_text(entry_text)
        assert e.get_text() == entry_text
        e.set_stamp()
        assert e.get_created() != e.get_stamp()

class JournalTest(UTest):
    # we'll write this shortly, before Journal!
    pass

if __name__ == '__main__':
    EntryTest()
    JournalTest()

Run urun.py from the command line or from your editor and the result:

# /www/lib/parlez/test% urun.py
./utest_journal.py: EntryTest:

No tracebacks indicates successful test(s).

In part five of this series we'll start to write the HTML (remember, this article series is apparently about web development with QP) and other user interfaces for our Entry object.

Basic Data Elements

As discussed in part one, this tutorial / web application project will result in a basic weblog or on-line journal application. Lets break down a weblog into its most basic data elements:

1. A weblog is a collection of writing, generally presented in chronological fashion. A weblog could be considered a diary or journal, so lets use the term Journal to describe its function.
2. A journal usually, but not always, represents the thoughts and opinions of a single author.
3. Each item in a journal can be considered an article or a post - lets use a more generic term and call each item in the journal an Entry. Entries are typically short bits of text so lets enter and store them as such. Each entry may have a title, and may include other information including dates relating to when the Entry was created, made available to readers, or changed -- but the principal information is the entry itself.

Our first classes

Turning to Python then, we could easily represent Entry as:

class Entry(object):
    title = None
    text = None
    created = None

We could then use the class:

e = Entry()
e.title = 'Python Web Application Diary, Part Three'
e.text = 'Hello, Bruce, my name is Bruce.'
created = datetime.datetime.now()

That was pretty simple, no? Simplicity can be both a boon and a pain in the butt, and experienced developers will recognize at least two significant problems with our still too-simple Entry object:

1. There is no way of easily persisting this data (saving it so that its available later when we need it)
2. The current design doesn't warn or otherwise prevent someone from intentionally or accidentally storing data we don't expect, such as:

e = Entry()
e.title = datetime.datetime.now()
e.created = 'Python Web Application Humour'

Kicking Entry Up a Notch - Persistence

Lets first look at the issue of persistence. Keeping our journal entries around for future display (or edits) could be done by:

• Saving the data into individual files
• Saving the data in a relational (SQL) database such as Postgres, Oracle, MySQL or MS SQL Server

class Entry(object):
    title = None
    text = None
    created = None

class Entry(PersistentObject):
    title = None
    text = None
    created = None

Where Code Lives

I presented in part one some of the choices I've made for my own system configuration as it pertains to file layouts and hierarchies.

QP is able to manage multiple applications, and requires only a little conformity to make that happen. Forget where applications should live? Issue the command qp -h.

/www/lib/Parlez/
                bin/
                doc/
                lib/
                    test/
                    ui/
                        test/

/www/lib/parlez -> /www/lib/Parlez/lib

mkqpapp.py ~/qp_sites/blog 8011

------------------------------------------------------------
blog application created in /usr/home/mw/qp_sites/blog

To start and stop blog issue the commands::

    qp blog start (or use qp -u blog)
    qp blog stop  (or use qp -d blog)

For more qp command line options::

    qp -h

To view the running application, visit::

    http://localhost:8011/

CHANGES
README
__init__.py
bin/
doc/
slash.qpy
test/
ui/
    test/
var/

qp -u blog

It worked!

The blog application lives at /usr/home/mw/qp_sites/blog.

The Project

The project I have in mind is to redevelop the blog software that powers my own personal site, http://mikewatkins.ca/. You are soaking in it right now, if you are reading this in early June 2007.

Why bother? While there are many blog applications out there, what I want at the end of this are a set of objects and UI components that can be integrated into other QP applications.

I've wanted to move off of PyBlosxom for some time now, but with something else always on my plate, doing so has never been a priority for me. Its the prospect of killing two birds with one stone -- wiring up a simple web log or journal, and documenting it tutorial style -- that is pushing me forward now.

Before Going Further

QP's developers rather unabashedly adopted some preferences which are built into the system and its design. It was designed to run on Unix-like systems. While some enterprising people have looked at running it on Windows, if you are a Windows-only or Windows-sometime developer, you might want to search and read the QP mailing list archives first before venturing forth.

First Steps

Today's installment for readers is to install QP and related components. But before we get there, first a word on my environment so that subsequent notes make sense.

My development, test and production environments have similar file system layouts and installed software. Of note to us in this tutorial you'll want:

• Python 2.5.x installed
• Python Imaging Library (PIL)
• HTMLTidy (will be used eventually)
• Perhaps a component or two more as we go through this.

I install all Python related tools into the default locations; tools and libraries and non-QP applications that I write are located in:

/www/lib - added to PYTHONPATH (in your .profile or .cshrc)

QP by default looks along certain paths for applications to manage (start, stop, restart, interact with, view log). This allows you to have user-space apps which QP can see with no re-installation of the base software, as well as system or privileged user applications. First lets look at the command line help:

%qp -h
Usage: qp [options] [start|stop|restart] [site]*

Control the servers for the sites in your qp installation.
No options is the same as "--status".
"stop" is the same as "--stop".
"start" is the same as "--start".
"restart" is the same as "--stop --start".

If "--stop" and "--start" are both present and the durus server
is already running, only the web server is restarted.

The script searches for sites in the qp.sites package, which
in turn searches through the following list, by default.
    ${QP_SITES}
    ~/qp_sites
    ~/.qp_sites
    /var/qp_sites
    /usr/local/lib/python2.5/site-packages/qp/sites

Options:
  -h, --help           show this help message and exit
  -u, --start          Start the durus and web servers.
  -d, --stop           Stop the durus and web servers.
  -q, --quickrestart   Quick restart of the web server(s) of the named sites.
  -v, --status         Show the current status of the durus and web servers.
  -c, --configuration  Show the site configuration.
  -l, --log            tail -f the log for the last named site.
  -i, --interact       Open an interactive session with the last named site.

QP applications other than demo apps or quick one-offs or test apps go:

/www/qp_sites

And I've created a symlink in /var that points to qp_sites:

cd /var
ln -s /www/qp_sites qp_sites

I put demo apps / quick one-offs / test apps in:

~/qp_sites

Also under /www for convenience:

/www/docs - a collection of documentation that I refer to
/www/python - symlink to /usr/local/lib/python2.5/

All the library work I do is contained in a Python package called "parlez". The development files live in /www/lib/Parlez, and a symlink to its lib subdir is created in /www/lib. Parlez contains:

/www/lib/Parlez         - including CHANGES, LICENSE, README
/www/lib/Parlez/bin     - related scripts
/www/lib/Parlez/doc     - documentation
/www/lib/Parlez/lib     - the Parlez package itself
/www/lib/Parlez/site    - a demo application

In /www/lib I create a symlink:

@ln -s /www/lib/Parlez/lib parlez@

Install QP and related packages

We want to install Durus, an object database closely related to QP but which can be used quite independently of QP, as well as QPY, QP, and Dulcinea. More on each of these later. Install them now, in the following order:

• Durus - a Python object database similar to ZODB
• QPY - Python-centric "templating" or if you prefer, untemplating
• QP - A web framework for easy publishing of "objects" to the web
• Dulcinea - pre-made components that work with QP and Durus
• Sancho - A unit testing suite

All of the software components listed above are available at http://www.mems-exchange.org/software/ under an open source license.

Once installed, run and play with one or more of the three QP demo applications provided:

qp proto start

or start proto like this:

qp -u proto

See what's running by issuing qp with no parameters:

%qp

proto  durus[61859]:localhost:7002 web[61860] http::8002 as_https:localhost:9002 https:localhost:10002
sr     durus:down web:down
recipe durus[60590]:localhost:7005 web[60591] http::8005
contact web[54418] http::8080
test   web:down
hello  web:down

And then visit http://localhost:8002/ to explore the prototype application.

Once you've looked at the basic QP demos and peeked at some of the underlying code to grasp how simple this all is going to be, then see what Dulcinea brings to the table by way of pre-made components and user interface bits. In your ~/qp_sites subdirectory, create a link to the Dulcinea demo application and run that.

In our next installment, we'll create some basic objects, introduce Sancho unit testing, and write a bloxsom to "Parlez" conversion script so we have some real data to work with going forward.