Using the SQLObject Support package
===================================

:Author: Sidnei da Silva
:Contact: sidnei@redesul.com.br
:Date: $Date: 2004-08-24 20:30:19 -0700 (Tue, 24 Aug 2004) $
:Version: $Revision: 1.1 $

This file describes how to use the ``sqlos`` package to use
SQLObject together with Zope 3.

Currently we're using a ``SQLObjectContainer`` object, that behaves
like a normal folder, except that its contents doesn't come from the
ZODB, but instead, they come from an SQL connection, through
SQLObject.

Given the pluggable nature of Zope 3, it was quite easy to integrate
it with SQLObject. Originally, SQLObject had its own ``DBConnection``
objects, that handle all the operations with the SQL backend, and even
some other backends, like ``DBM``. These connection objects implement
the ``DBAPI``, and work over a SQL cursor that is generated on the
``makeConnection`` method of the DBConnection-based classes. Each
database connection implements a given set of methods to generate
queries and convert data types. To allow using a connection from Zope
3 in SQLObject, we used an ``adapter``, which adapts a
``IZopeDatabaseAdapter`` utility service to the same interface that a
DBConnection object implements. This adapter just subclasses the
``*Connection`` classes from SQLObject (eg: MySQLConnection),
overriding the ``__init__`` and ``makeConnection`` method to
initialize an instance variable with a reference to the ZopeConnection
object and to return a ZopeCursor instance respectively.

For the SQLObjectContainer class, we made it so it is possible to
choose a set of class names that will be available inside that container.

Based on this, the methods of the IContainer class were implemented so
as all operations are done using SQLObject methods. For example, the
``keys`` method does a select query and returns the available ids, the
items method uses the SQLObject class ``select()`` method to generate one
object for each row in the database and return them.

At this point, we have an object that behaves just like a folder, and
its flexible enough that you can make their objects come from any
connection and any registered class you wish.

Making SQLObject-based objects editable inside Zope
---------------------------------------------------

Now, for making the SQLObject's editable there are only a few more
steps. We already created an interface called ISQLObject, with all the
methods and declared that the SQLObject class implements this
interface. The ``class chooser`` used on the SQLObjectContainer uses
this interface to query the registered classes for the ones that it is
able to create. Let's see how to create a SQLObject from scratch now.

See this example::

 from zope.interface import implements
 from SQLObject import *
 from zopeproducts.sfbr.interfaces import IPerson

 class Person(SQLObject):

     implements(IPerson)

     _columns = [StringCol('username', length=20, notNull=1),]

Here you can see that there is almost nothing special needed for
making the object be recognized by Zope 3. In fact, if you remove the
``implements(IPerson)`` line, it is **exactly** the same code you
would use for a standard SQLObject class (Note: if you want, you can
declare that an object implements a given interface in a separate
file, or even in ZCML).

Let's take a look at what the IPerson interface says::

 from zope.schema import TextLine
 from zope.interface import Interface, Attribute

 class IPerson(Interface):

     id = Attribute('Id')
     username = TextLine(title=u'Username',
		     description=u'The name of the user')

As you can see, there's not much here either. We're just saying that
classes that implement IPerson will have an ``id`` attribute (all
SQLObject instance have that by default) and a ``username`` attribute,
which happens to be a ``TextLine`` (well see more about that later)
with a label and a description.

This interface is used to auto-generate the edit forms inside Zope
3. Let`s see what is needed to do that::

 <zopeConfigure
    xmlns='http://namespaces.zope.org/zope'
    xmlns:browser='http://namespaces.zope.org/browser'>

   <browser:editform
      schema="zopeproducts.example.interfaces.IPerson"
      name="edit.html"
      menu="zmi_views"
      label="Edit a Person"
      permission="zope.ManageContent"
      />

   <content class="zopeproducts.example.person.Person">

     <factory
	 id="Person"
	 permission="zope.ManageContent"
	 title="Person"
	 description="A Simple Person" />

     <require
	 permission="zope.ManageContent"
	 interface="zopeproducts.example.interfaces.IPerson"
	 set_schema="zopeproducts.example.interfaces.IPerson"
       />

   </content>

 </zopeConfigure>

There are a few things that you need to know about this piece of
ZCML:

First, the <browser:editform /> part is what does generate the
automatic editform for the Person class. As you can see, we don't
reference the class explicitly, but instead, we are saying that for
the ``IPerson`` schema (a schema is just a flavor of interface), there
will be an auto-generated editform called ``edit.html`` and it will be
registered with the menu ``zmi_views``. <browser:editform /> is a
``directive``, which is also registered using ZCML and is created as
part of the ``metaconfiguration`` of Zope 3. I'll not enter this into
detail now. All you need to know is that this is a ``special view``
that spits out an edit form in HTML based on the schema you defined.

Second, the <content class="...."> directive is used to register some
information about the class, like what factory will be using for
creating instances of it and what are the permissions needed to do
stuff with this class. The <factory> stuff is not completely plugged
in at the moment, so let's take a look at the <require> directive:

As you can see, there are 3 attributes on the directive. The
``permission`` attribute specifies the permission that is
required. The ``interface`` attribure specifies what is the interface
that is protected by this permission. All the methods and properties
defined on this interface are protected for reading by this
permission. And last, the ``set_schema`` attribute specifies what is
the interface which is going to be protected for **setting**
attributes on the instances of this class. This is needed for the
auto-generated form to work. If you don't specify a permission to
protect ``set_schema`` on the interface, you won`t be able to modify
the instance.

Now, the last thing needed is registering the class with the
``ClassService``, which was created temporarily because there wasn't a
service that could do what we wanted. The service that is more closer
to what we need is the Factory Service, but Jim said he was going to
change it soon, so its better to have our own service for now. Later
we can merge the functionalities we need into the Factory service.

For registering our class with the class service, all you need is
this::

 <zopeConfigure
    xmlns='http://namespaces.zope.org/zope'
    xmlns:classService='http://namespaces.zope.org/class'>

   <classService:registerClass
	 component="zopeproducts.example.person.Person"
	 name="Person"
	 />

 </zopeConfigure>

I will not enter in details about how this work either. You just need
to know that this registers the person class under the name
``Person``, so that's what you will see on the SQLObjectContainer edit
form as the class available for choosing.

XXX Write about SQLObjectAuthSource.

Notes about transaction interoperability
----------------------------------------

SQLObject, starting on version 0.6, has a new feature called ``lazy
updates``. This feature allows for batching changes in a controlled
way so that only one ``UPDATE`` statement is run for a set of
changes. This is particularly useful for our Zope integration, because
when you edit an object, all of it's attributes are passive of being
changed. This can potentially cause a big set of ``UPDATE`` queries to
be fired. The triggering of the ``UPDATE`` query is done manually, by
calling either ``sync()`` or ``syncUpdate()`` on the SQLObject-based
object (``sync()`` refetches the the data from the database, which
``syncUpdate()`` does not).

Of course, for a more transparent integration, it shouldn't be
required for the user to do that, but instead, when Zope (ZODB) does a
transaction commit (usually on request boundaries, but can happen at
any given time), it ``sync()`` should be called for any objects marked
as ``dirty``, so that their ``UPDATE`` queries are run. Also, the
order that the queries are to be issued should follow the order which
objects get dirty, so that any constraints/triggers can do their job
accordingly.

Given the current state of ZODB's transaction machinery, there is a
relatively non-intrusive change that can be done to ``sqlos`` to
achieve this goal. Here we will explain the implementation.

In Zope, each request is served by a ZServer thread. For each request,
a transaction (bound to a thread id) is created. When the request
ends, the transaction gets commited.

ZODB 4 (currently postponed for lack of time) introduced the concept
of ``joining`` a transaction. For objects to join a transaction, they
must either implement ``IDataManager``, or have a ``_p_jar`` attribute
which implements IDataManager. When the ``main`` (or, original)
transaction is about to commit, it's ``prepare()`` method is called
first, then ``commit()`` is called. When the transaction gets aborted,
it's ``abort()`` method is called. The main transaction makes sure
that for every ``IDataManager`` that joined the transaction, it's
``prepare()``, ``commit()`` and ``abort()`` method are called. The
order of the registered objects is given by calling the ``sortKey``
method on the IDataManager if available, or by
``id(<IDataManager>)``.

Our solution consists of:

  - Create a SQLObjectTransactionManager that implements
    IDataManager. This object will keep track of the dirty objects for
    a transaction.
  - Anytime a SQLObject-based object gets its ``dirty``
    bit set, the object is registered with the current
    SQLObjectTransactionManager.
  - If an object gets it's dirty bit set to ``False``, it may mean
    that this object's ``sync()`` method has been called. We *could*
    call ``sync()`` on all the dirty objects registered before, but
    this would increase complexity a lot. For now, we just recommend
    *not* calling ``sync()`` or ``syncUpdate()`` manually inside
    transactions.
  - SQLObjectTransactionManager needs to have a ``sortKey()`` that
    returns a value smaller than ZopeDBTransactionManager's (which
    currently doesn't implement this method, so
    ``id(<ZopeDBTransactionManager>)`` is used). This is so that the
    ``UPDATE`` queries get sent before the RDB cursor's ``commit()``
    is called.
  - Create a SQLOS class that subclasses SQLObject to override
    ``dirty`` and enable lazy updates by default.
