Professional Documents
Culture Documents
Framework
Table of Contents
Introduction 0
Getting Started 1
API Reference 2
Models 2.1
Schemas 2.2
Mixins 2.3
Collections 2.4
Querying 2.5
Columns 2.6
Databases 2.7
Connections 2.8
Integrations 3
Pyramid 3.1
Qt 3.2
Cookbook 4
2
ORB Framework
ORB Framework
The ORB framework is a database ORM designed to easily create object-oriented APIs for
various database backends.
This is an open-source project released under the MIT license, the code for the framework is
hosted online at GitHub.
The core of the framework is the orb library, a Python package that provides the
implementations for the model structure and communication with the backend data services.
In addition to the core library, we're developing a number of useful middleware utilities to
help generate full applications for both web and desktop, which we will go into more in detail
later.
Introduction 3
ORB Framework
Getting Started
Installation
The main library for ORB is the orb-api package. This library can be installed using the
standard Python setuptools and pip .
Note: When you see an example start with >>> in the documentation, this just denotes
that the example is for the Python console. When you see an example start with $ , it
denotes that the example is for a command line or shell.
This will download and install the core Python code, and you can start building out your APIs
with that.
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.PasswordColumn()
This is the way a model looks in ORB. There are two kinds of models, a Table and a View.
Most commonly, you will be working with Table's, as these kinds of models support reading
and writing to a database or data-store.
Getting Started 4
ORB Framework
For this example, we have just defined a User class. This will create a standard Python
class which will register itself to the global orb.system , defining it's schema information for
syncing to various backend databases.
We've defined 3 columns for our user -- the IdColumn (every model is required to have an id
column associated with it) called id , a StringColumn called username , and a
PasswordColumn called password . We'll go into more detail about the differences between
To do that, we need to define a new database. For a list of supported databases, take a look
at the connection types page.
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
That's it! We've just created a simple database, using the standard SQLite Python adapter.
This will create a file called example.db in the directory where you just executed this code.
Note: SQLite is a lite implementation of SQL. As such, it will not have full feature
functionality. We will document what adapters more advanced features are supported in
as we get to them and continue to improve the framework.
ORB supports model access to multiple databases and references between them. To do
this, there are a number of ways that you can associate a database to a model. The most
common of which, when you have a single database/datastore to access, is to just activate
the database. Then it will be globally available for all of your models.
Getting Started 5
ORB Framework
Done! We will go into more detail about this later. But you can now work with your database.
In this example, we have instantiated 2 User records. You can provide a model's constructor
a dictionary for the column name and its corresponding value, and this will setup the values
for your new record.
Getting Started 6
ORB Framework
So, we have now created 2 records, but at this point, they only exist in memory -- we have
not actually stored them to our backend data store. The way ORB works, is you can create
or retreive records, modify them, and when you are ready, save them back to the database.
Only changes will be stored, and if no changes were made, then no database calls are
made!
The save function will commit your changes to the backend. You can see in the example
above, before the save, the user_a instance had no ID, and afterwards, it now has one.
When the record saves for the first time, it will create a new instance and store it, while
subsequent saves will update the record instead.
In this example, we have fetched the user whose id is 1 from the database, which returns
the data that we stored from user_a .
Note: There are 3 ways to initialize a model, and at this point we've seen 2 -- you can
initialize with a dictionary of column/values, you can initialize with an ID, and you can
initialize with nothing. If you initialize with nothing, like User() , it will create a new in-
memory instance with no pre-defined values for columns.
Retrieval failure
If you attempt to retrieve a record from the database and it does not exist, a RecordNotFound
error will be raised:
Getting Started 7
ORB Framework
This is something to be aware of, so if you are unsure if a record exists or not, then you
should wrap this retrieval in a try/catch block.
Creating an Index
Often times, being able to retrieve models by data other than the ID is useful. If I wanted to
lookup a user based on their username or email for instance, there is an easy construct for
doing this in ORB.
Indexes provide an easy way to define common ways to lookup data from a database. They
also can provide information to the backend to optimize how that data is stored because we
now know that it is important to use for retrieval.
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
We have added the byUsername index to the User model. What this will do is generate a
class-level method that you can use to lookup a model by simply passing in the username.
Getting Started 8
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
As you can see, we've now added two new columns, first_name and last_name , and a
new index byName . For this index, we have specified that it is not unique (since you can
have many John Doe's). Instead of getting a record back from this new index, we will now
get back a collection.
Getting Started 9
ORB Framework
>>> # first, let's update our 'john' user to have a first and last name
>>> u = User.byUsername('john')
>>> u.set('first_name', 'John')
>>> u.set('last_name', 'Doe')
>>> # saving the record now will update the existing user record
>>> u.save()
Creating Relationships
One-to-Many
One of the most important things that you need to be able to do with an ORM is create
relationships between your models. The way this is done in ORB is by creating a reference
column on one of your models.
Let's open up the intro.py file again. This time, we're going to add a new model called
Address , and relate it back to the user.
Getting Started 10
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
class Address(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
name = orb.StringColumn()
street = orb.StringColumn()
city = orb.StringColumn()
state = orb.StringColumn()
zipcode = orb.IntegerColumn()
The new Address model should be a little familiar to you by now. The new piece of the
puzzle is the ReferenceColumn that we added to it. What this does is creates a relationship
between the Address and User models, where an Address record will store the User
reference information in the user column.
For instance:
>>> u = User.byUsername('john')
>>> address = Address({'name': 'Home'})
>>> address.set('user', u)
>>> address.set('street', '555 Main St.')
>>> address.set('city', 'Anywhere')
>>> address.set('state', 'CA')
>>> address.set('zipcode', 55555)
>>> address.save()
>>> print a.id()
1
Getting Started 11
ORB Framework
We have now created a new Address record, and related it to the user 'john'. If I retrieve
this address now, I can see what user it is related to:
By default, this reference is a one-to-many relationship, in that one user can have multiple
addresses.
One-to-One
One-to-one relationships are just as easy to create. All you need to do is to add a 'Unique'
flag to the definition of the ReferenceColumn . This will tell the system that only one reference
of each unique record is allowed. For instance:
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
class Preference(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User', flags={'Unique'})
notifications_enabled = orb.BooleanColumn()
class Address(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
name = orb.StringColumn()
street = orb.StringColumn()
city = orb.StringColumn()
state = orb.StringColumn()
zipcode = orb.IntegerColumn()
Getting Started 12
ORB Framework
This will now enforce that a user record has only one Preference that it can be related to.
>>> u = User.byUsername('john')
>>> p = Preference()
>>> p.set('user', u)
>>> p.set('notifications_enabled', True)
>>> p.save()
Collectors
So we have preferences for a User, and they can have multiple addresses. But right now, we
still have to know the ids of those records to retrieve them. The easiest way to interrelate
these models more fully is to assign some collectors to them. A collector is an object that will
define how to select a collection of records from the database, given a set of criteria.
Reverse Lookups
Reverse lookups will collect records from the database by looking at a reference column in
reverse. What I mean by this is instead of asking for an Address and then getting it's User ,
I can ask for a User and then have the collector find it's related Address records.
Getting Started 13
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
addresses = orb.ReverseLookup(from_column='Address.user')
preferences = orb.ReverseLookup(from_column='Preference.user', flags={'Unique'})
class Preference(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User', flags={'Unique'})
notifications_enabled = orb.BooleanColumn()
class Address(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
name = orb.StringColumn()
street = orb.StringColumn()
city = orb.StringColumn()
state = orb.StringColumn()
zipcode = orb.IntegerColumn()
Because the Address.user column is not unique, and the relationship is one-to-many, what
is returned from calling addresses is a collection. Because the Preference.user is unique,
however, an individual record is returned.
Getting Started 14
ORB Framework
Pipes (Many-to-Many)
Another type of collector is the ORB solution for the many-to-many relationship called a
Pipe. We have taken the explicit approach for many-to-many relationships, requiring you to
define an intermediary model which you will "pipe" through for your relationships.
If we turn to the intro.py file again, we will introduce the concept of "user groups". For this,
one user may be a part of many groups, and one group may have many users. This use
case is what defines the many to many relationship.
Getting Started 15
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
addresses = orb.ReverseLookup(from_column='Address.user')
preferences = orb.ReverseLookup(from_column='Preference.user', flags={'Unique'})
groups = orb.Pipe(through_path='GroupUser.user.group')
class Preference(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User', flags={'Unique'})
notifications_enabled = orb.BooleanColumn()
class Address(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
name = orb.StringColumn()
street = orb.StringColumn()
city = orb.StringColumn()
state = orb.StringColumn()
zipcode = orb.IntegerColumn()
class Group(orb.Table):
id = orb.IdColumn()
name = orb.StringColumn()
users = orb.Pipe(through_path='GroupUser.group.user')
class GroupUser(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
group = orb.ReferenceColumn(reference='Group')
You will see from this example that we have created 2 new models -- Group and
GroupUser . The GroupUser table will act as the intermediary between Group.users and
User.groups pipes.
Getting Started 16
ORB Framework
You'll notice that for these examples, we first created a Group record, and then associated it
to the pipe whereas for the Address example we created the address record setting the
reference column to a User . The add method on a collection is a convenience method for
creating the intermediary record. You could have also done it this way:
Removing Records
Getting Started 17
ORB Framework
We've discussed most of the core functionality of ORB and general ORM's -- create, read,
update, relationships. The last topic we'll discuss in this general overview is how to delete
records from the database.
Using our last example, we now have 2 users (john and jane), two groups (admins and
editors) and an address and a preference.
For discussing deleting records, we'll cover it in 2 sections -- removing an individual record
and unlinking a record from a many-to-many relationship.
In the first example, we directly accessed the Address model for id 1 and deleted it. This
removed the record from the data store, and consequently john no longer had that record in
his collection. After the record was deleted, it's ID was cleared since it no longer exists in the
data store -- although you do still have access to the in-memory record instance.
In the second example, we're actually deleting the intermediary record between our user and
group models...not the user or group themselves. To do this, you could have selected the
particular GroupUser you wanted to delete and delete it directly, but there is the
convenience method remove for a collection that will do that for you. Calling
john.groups().remove(editors) didn't delete john or editors, it instead deleted the
GroupUser record whose user was john and whose group was editors.
Getting Started 18
ORB Framework
What's up next
So that covers all of the basics of the ORB framework...but there is so much more to the
system. To cover it all in a single page is too much, so we have broken down the in-depth
instruction on the API Reference page, and have more real-world use case walkthroughs in
the Cookbook.
Getting Started 19
ORB Framework
API Reference
Models
ModelMixins
Database
API Reference 20
ORB Framework
Models
As you've seen from the getting started page, a Model class type defines the objects that
drive an ORB API. Most of the functionality of the Table class that you have been working
with is actually derived from the base Model class. In addition to tables, there is also a
View model type in ORB.
Note: All examples will build off the Getting Started models that we have already
defined.
The Table class is a model that allows for read and write capability, and will be what you
use to store records and retrieve them later.
The View class is a read-only model that allows you to define multiple columns to be pre-
joined together in the database or store for easy querying and access later on.
Basics
The basics for any data-driven system is CRUD -- Create, Read, Update and Delete. We
have already touched upon this in the step-by-step introduction to ORB in the gettting
started page, but will go into it in more detail now.
Creating records
There are a few different ways that you can do this. The most straightforward way is to
instantiate a new model object, set its column values, and then save it.
1: Model constructor
Models 21
ORB Framework
When to use one way or another is up to you, based on what makes sense for the
application. To ORB, either way works just as well.
2: Model.create [classmethod]
If you do know what the properties you are going to create your record with before hand, you
can also use the create class method, which is a convenience method to the second
approach:
The create method will create a new instance of the model classin memory, populate the
column fields from the given dictionary, and then save the record to the database and return
it.
3: Model.ensureExists [classmethod]
The above two examples will create new models, without care for what already exists. For
instance, if you have a Unique constraint on a column and try to create a second record that
conflicts, the system will raise an orb.errors.DuplicateEntryFound error. If what you want is
to make sure that a record exists in the database with a certain set of properties, you can
use the ensureExists classmethod instead. This will first look to see if a record of a given
model already is defined in your database and return it to you, otherwise, it will create the
new record first and then return it.
Models 22
ORB Framework
As you can see from the above example, the first call to ensureExists returned the already
created user for id 1. The second call created a new user with id 6.
One important difference to note here: when using ensureExists , you are providing 2 sets
of column/value pairs. The first set is what will be used to verify uniqueness -- so in this
example, we want to make sure a User of username 'john' exists. If that user does not
exist, then create it with the additional default values. If we provided both username and
password, then it would verify that combination is unique and in the database, but that is not
what we care about since the user may have changed their password.
4: Collection.add/Collection.create
The final way to create new models is through a collection. This is a way to create models
and relationships when working with reverse lookups and pipes, but we will get into that
more in the collection page.
Reading Records
As with creation, there are a number of ways to retrieve records. Two of the more common
ways are retrieval via id and retrieval via index, as we discussed in the getting started page.
The most general way however is to use ORB's query syntax, but that is so powerful it
deserves it's own page.
1: Fetch an ID
Models 23
ORB Framework
That's it. Simply provide the ID of the record that you are looking for to the constructor and it
will go fetch it for you. If the record does not exist in the database, the RecordNotFound error
will be raised:
2: Model Indexes
Indexes provide a structure that allows common lookups and queries to be pre-defined for
both database lookup efficiency, and code reduction. These objects are added to the Model
class definition and are callable as classmethod 's returning either a record or collection
based on their uniqueness.
We will go more in-depth on indexes in the index page, but this is the basic idea. These
methods will not throw an error for a record not found. They will return None for nothing
found instead:
The difference here is that we're not directly fetching a known resource (as with lookup by
ID) that we expect to exist. We are returning the results of a pre-defined query that we don't
know if it exists or not.
3: Model.all
Models 24
ORB Framework
Another option for retrieval is to simply return all the records that exist for the model. This
isn't recommended for large data sets, but can be very useful if you want to cache things like
statuses that are relatively known and small enough.
4: Model.select
The most general way to retrieve records however, is to use the Model.select class
method. This will allow you to build a query and filter out records that you want to retrieve,
limit the results of your search, order the records, and define specific columns to fetch as
well. All other systems build on top of this generic one -- Indexes simply pre-define the query
information to provide to the model selection for instance.
We will go into more detail on the querying options in the query documentation, but if we
want to show how the byUsername and byName indexes work as an example, we could fetch
the same information by doing:
Updating Records
Models 25
ORB Framework
As we explored in the getting started page, you can update records by calling the set
method, however there are a couple of other ways to do it as well:
1: Model.set
2: Model.update
The second way to update a model is in bulk, passing in a key/value pair of columns. This
will still call the internal setter methods if you want to do custom work when changing a
column.
Note that you still need to save after you have modified column values. Calling set or
update will only change the values that are stored in memory, not in the database, until
save is called.
Deleting Records
The final operation for all CRUD ORMs, deleting records. In ORB there are really 2 ways to
remove records -- individually, or in bulk from a collection.
1: Model.delete
The easiest way to remove a record from the database is to simply call delete on it.
Models 26
ORB Framework
2: Collection.delete
The second way to remove records is through a collection. This can be done via collectors,
or from a direct selection:
The response from deleting records is the number of records that were removed.
Models: Events
As operations are happening throughout the orb system, events will be triggered and can be
connected to for performing custom operations.
All events can set the preventDefault property to True to override the default functionality.
Model.onSync [classmethod]
The onSync method will be called during the Database sync method, directly after the
model has been created or updated in the datastore. This method is commonly used to
define default values for a database.
Models 27
ORB Framework
import orb
class Status(orb.Table):
id = orb.IdColumn()
code = orb.StringColumn(flags={'Unique'})
name = orb.StringColumn()
@classmethod
def onSync(cls, event):
for default in (('in_progress', 'In Progress',
'approved', 'Approved',
'rejected', 'Rejected')):
code, name = default
cls.ensureExists({'code': code}, defaults={'name': name})
This code, when synced to different databases, will ensure that the default statuses exist in
each environment.
import orb
class Comment(orb.Table):
id = orb.IdColumn()
text = orb.TextColumn()
created_by = orb.ReferenceColumn(reference='User')
parent = orb.ReferenceColumn(reference='Comment')
Model.onChange
Models 28
ORB Framework
The onChange event is fired whenever a column value is changed. This will be done at
runtime, so this will happen before the changes are saved to the database. You can use this
to provide additional changes and verification on individual columns.
import orb
from orb.errors import ValidationError
class Ticket(orb.Table):
id = orb.IdColumn()
status = orb.ReferenceColumn(reference='Status')
title = orb.StringColumn()
description = orb.TextColumn()
Model.onDelete
The onDelete event is fired whenever a model is removed from the database. This will be
called before the delete actually occurs, and you can set the preventDefault in the event to
block the normal process from occurring. (You can also always raise an error)
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
active = orb.BooleanColumn()
def onDelete(event):
self.set('active', False)
self.save()
event.preventDefault = True
In this example, we override the default functionality to disable deleting a User record, and
instead just deactivate them.
Models 29
ORB Framework
The final events for the Model class are the onInit and onLoad events. These events are
fired when a model is initialized. The onInit event is triggered when a new model is
created, while the onLoad event is triggered when a model has been loaded from the
database.
import orb
class Comment(orb.Table):
id = orb.IdColumn()
created_by = orb.ReferenceColumn(reference='User')
text = orb.TextColumn()
In this example above, we can use the onInit callback to assign the default created_by
user to a current user. (Note, this assumes there is a User.currentRecord method).
Inheritance
As mentioned in the introduction, ORB is an object-oriented framework. One of the more
powerful aspects of the system is it's inheritance structure -- which will allow you to inherit
data between models easily.
Models 30
ORB Framework
import orb
class Asset(orb.Table):
id = orb.IdColumn()
code = orb.StringColumn(flags={'Unique'})
display_name = orb.StringColumn()
parent = orb.ReferenceColumn(reference='Asset')
project = orb.ReferenceColumn(reference='Project')
type = orb.StringColumn(flags={'Polymorphic'})
children = orb.ReverseLookup(from_column='Asset.parent')
dependsOn = orb.Pipe(through_path='Dependency.target.source')
dependencies = orb.Pipe(through_path='Dependency.source.target')
class Dependency(orb.Table):
id = orb.IdColumn()
source = orb.ReferenceColumn(reference='Asset')
target = orb.ReferenceColumn(reference='Asset')
class Project(Asset):
budget = orb.LongColumn()
supervisors = orb.Pipe(through_path='ProjectSupervisor.project.user')
class ProjectSupervisor(orb.Table):
id = orb.IdColumn()
project = orb.ReferenceColumn(reference='Project')
user = orb.ReferenceColumn(reference='User')
class Character(Asset):
polycount = orb.LongColumn()
is_hero = orb.BooleanColumn()
In the above example, we have defined a number of tables. The key points to the inheritance
structure are 2 points:
To define an inheritance structure in the ORM, you simply need to inherit one model from
another, as the Character and Project models inherit from the Asset model.
Models 31
ORB Framework
>>> Character.schema().dbname()
'characters'
>>> Project.schema().dbname()
'projects'
>>> Project.schema().inherits()
'Asset'
Polymorphic column
To have the system be able to automatically track the source class, you should define a
polymorphic column in the base table. This will store the model name when it saves, and
then use that type when inflating.
Advanced Usage
The getter and setter methods can be overridden to provide additional, custom functionality.
If we modify the intro.py file to read:
Models 32
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
addresses = orb.ReverseLookup(from_column='Address.user')
preferences = orb.ReverseLookup(from_column='Preference.user', flags={'Unique'})
groups = orb.Pipe(through_path='GroupUser.user.group')
@username.getter()
def _get_username(self):
print 'getting username'
return self.get('username', useMethod=False)
@username.setter()
def _set_username(self, username):
print 'setting username'
return self.set('username', username, useMethod=False)
Models 33
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
first_name = orb.StringColumn()
last_name = orb.StringColumn()
groups = orb.Pipe(through_path='GroupUser.user.group')
@orb.virtual(orb.StringColumn)
def displayName(self):
return '{0} {1}'.format(self.get('first_name'), self.get('last_name'))
@orb.virtual(orb.Collector)
def allGroups(self, **context):
groups = self.groups(**context)
groups.add(orb.Group.byName('Everyone'))
return groups
In this example, we have added 2 virtual methods -- displayName and allGroups . These
will now exist within the schema, however will not exist in the database. By default, virtual
methods are ReadOnly , so if you try to set the display_name value, it will raise an error.
To support setting virtual columns, you can use the setter decorator like so:
Models 34
ORB Framework
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
first_name = orb.StringColumn()
last_name = orb.StringColumn()
groups = orb.Pipe(through_path='GroupUser.user.group')
@orb.virtual(orb.StringColumn)
def displayName(self):
return '{0} {1}'.format(self.get('first_name'), self.get('last_name'))
@displayName.setter()
def setDisplayName(self, name):
first_name, last_name = name.split(' ', 1)
self.set('first_name', first_name)
self.set('last_name', last_name)
@orb.virtual(orb.Collector)
def allGroups(self, **context):
groups = self.groups(**context)
groups.add(orb.Group.byName('Everyone'))
return groups
Schema as Mixin
Another common paradigm (and one we will get into later with automatic schema
generation) is to have one class be the base schema, and the other be the custom model.
You can achieve this by using the ModelMixin class. We could redefine the User class
definition as such:
Models 35
ORB Framework
import orb
class UserSchema(orb.ModelMixin):
id = orb.IdColumn()
username = orb.StringColumn(flags={'Required'})
password = orb.StringColumn(flags={'Required'})
first_name = orb.StringColumn()
last_name = orb.StringColumn()
addresses = orb.ReverseLookup(from_column='Address.user')
preferences = orb.ReverseLookup(from_column='Preference.user', flags={'Unique'})
groups = orb.Pipe(through_path='GroupUser.user.group')
In this example, we can just override the username method since there will not be a conflict
with the column definition for the schema. Instead, we just redefine what the username
method does and it will automatically be connected.
When you use a mixin, the logic and schema is applied to other models at generation time,
but you cannot share base references to a mixin. This is because each column, while shared
in definition, is copied and unique in implementation per model.
Inheritance, on the other hand, will share the base schema. Internally, it will also store the
base table as its own data table, merging into inherited tables at query time. Doing this,
allows you to share references to base objects.
Models 36
ORB Framework
import orb
from orb import Query as Q
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
password = orb.StringColumn()
active = orb.BooleanColumn(default=True)
@classmethod
def baseQuery(cls, **context):
context = orb.Context(**context)
if not (context.where or context.where.has('active')):
return (Q('active') == True)
else:
return Q()
In this example, if there is no query already defined, or if the query does not already define
options for the active column, then we will default the query to only use active users.
Models 37
ORB Framework
Mixins
In addition to Models, you can achieve a lot of useful code reuse through the ModelMixin
class. This allows you to define base functionality and reuse it as a mixin to one or more
models later on.
import datetime
import orb
class AuthorshipMixin(orb.ModelMixin):
"""
Defines a re-usable class for tracking what user
created a record, and the time when it was created.
"""
created_by = orb.ReferenceColumn(reference='User')
created_at = orb.DatetimeColumn()
Mixins 38
ORB Framework
import orb
from .mixins import AuthorshipMixin
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
@classmethod
def currentRecord(cls):
return getattr(cls, '__record', None)
@classmethod
def setCurrentRecord(cls, record):
setattr(cls, '__record', record)
In this example, the AuthorshipMixin will add the created_at and created_by columns to
any model that wants to track it. Adding it to the Comment and Ticket models will let you
reuse the creation logic without having to re-code it, or define it as a base table.
Mixins 39
ORB Framework
Collections
At this point, you have worked with collections a little bit. This chapter will delve into them in
more detail. Collections are a group of records, which can be a representation of what
models to fetch from the database - or a list of records themselves.
Collection.records
Underneath the hood, this is actually calling the records method for iteration. The same
could have been written as:
Why is it important to know that? The first time a collection's records property is called, it
will decide if it has already been loaded from the database, or if it needs to go load itself.
Every subsequent call will return the data that was loaded previously.
Collection.iterate
Often times, this is a perfectly acceptable way to iterate through a list of users. Sometimes
however, particularly with large data sets, you will want to speed up the process by only
loading sub-sets of information at a time. You can do this with the iterate method:
Collections 40
ORB Framework
In the call to records , all the records of the collection are fetched first, then iterated
through.
In the call to iterate , we are actually batching the results. There is an optional batch
keyword argument to the iterator. As you loop through, and the iterator hits its end, it will
then go and select the next subset of data. The default size of the batch is 100 records, so
each 100 records a new query to the backend will be performed. You can modify your
sample size by just changing the batch keyword:
Pagination
Similarly to iteration, you can also limit the records you fetch via paging. Paging with ORB is
very easy -- just define the size of the page and which page you want to get. Assume we
have a database with 100 users, with IDs 1-100. If I were to query that using paging, it would
look like:
Collections 41
ORB Framework
If you want to pre-define your page size, you can do so in the all or select methods as
well:
Pre-defining the page size is passed via context, which we will go more into later.
The above would be the same way to represent page 2 in the paging structure.
Slicing
In addition to paging and manually setting start/limit, collections also support Python's slicing
syntax. Before a collection has fetched records from the backend, you can slice it down and
that will update the start and limit values accordingly.
Collections 42
ORB Framework
Sometimes, all you need are the values from a collection vs. full object models. You can
improve performance by only requesting the data that is needed for your system from the
backend. To do this, you can ask for specifically values -- or even just unique values --
through the collection.
If you only request a single column, then you will get a list containing the values. If you
request more than one column, you will get a list of list values back, where the contents of
the list is based on the order of the columns requested.
If you are looking at non-unique columns and want to just get a sense of all the unique
values with in a column, you can use the distinct method to fetch only unique entries:
As you can see, the values method will return all values for a column within your collection
-- including the duplicate first name of 'John'. Using distinct , we just get one instance per
value.
Collections 43
ORB Framework
Note:
If you request a reference by name from the collection, it will return an inflated model
object unless you specify inflate=False . If you request a field, you will get the raw
reference value.
Grouping
As we just showed, you can create a dictionary based on the returned data from the values
method. You can also use the built-in grouped method for a collection to group data via
multiple columns into a nested tree structure.
If you pass in the optional preload=True keyword, this method will fetch all the records in
the database first, and then group them together. The resulting collections within the groups
will be already loaded and no further queries will be made. This is useful if you are not
loading a large set of data.
If you choose preload=False (which is the default), then only the values of the columns will
be selected. The resulting collections within the group will be un-fetched collections with
query logic built in to filter based on the given column values that group them. This will result
in more queries, but smaller data sets, which could be useful for generating a tree for
instance that would load children dynamically.
Retrieving Records
Short of iterating through the entire record set, there are a few different ways that you can
retrieve a record from a collection.
Collections 44
ORB Framework
Often times, you will want to get the first and/or last record of a collection. Maybe you are
just looking at the last comment that was made in a thread, or the original edit made to a
record. The first and last methods will allow you to do this easily:
How does the system know what is first or last? If not specified, it will default to ordering the
record's IDs. First will return the lowest ID while last will return the highest.
This doesn't work for non-sequential IDs however, or if you want to sort based on some
other criteria. To manage this, simply provide the ordering information:
The ordered method will return a copy of the collection with a new order context (see the
Contexts page for more detail on ordering).
Collection.at
You can also lookup a record by it's index in the collection. You can do this using the at or
__getitem__ of the collection:
Collections 45
ORB Framework
Warning:
This is more expensive than using first and last because it will query all the
records and then return the record by index. You should only use this if you already
have, or are going to get, all the records.
In this case, the first time the users[i] is called, all the records are looked up. Every
subsequent request by ID will be against your already cached records, no more database
calls will be made.
Modifying Records
After a collection has been defined, there are a number of ways to modify the records.
Collection.refine
The most common thing to do is to refine selections. This will allow you to make selection
modifications for your record for reuse.
The refined collection will join the query of the base collection with any modifications and
return a new query.
Collection.ordered
The ordered method will update the order by properties for the collection's context and
return a newly updated collection set.
Collections 46
ORB Framework
Collection.reversed
The reversed method will invert the order by property and return a new collection set in
reverse order.
Editing Records
In addition to using collections as a means of selecting and retrieving records, they are also
used for bulk creation, editing and deletion.
Collection initialization
So far, we've seen two ways of accessing collections -- from the Model selectors ( all and
select ) and from collectors. You can also just create an empty collection.
When you create a new blank collection, it will be null by default (no information whatsoever)
What can you do with this? Working and iterating through these records in memory is really
not much different than using a regular list. However, you can use this to save these records
to your backend data store with a single save call now:
>>> users.save()
Calling the save method will create 2 new records in the data store.
Collection.add
Collections 47
ORB Framework
In addition to initialization, you can dynamically add records to collections using the add
method. One thing to note - this method is context aware, meaning based on where the
collection came from, it will behave slightly differently.
>>> users.save()
This example starts with a blank collection, and then adds users to a list in memory, and
then saves. This is functionally the same as initializing the collection with a list of records.
When you access a collection via a collector -- such as a ReverseLookup or Pipe , the add
method will actually associate the given record in the data store directly.
In these examples, the add method will dynamically associate the records through the
collectors in the data store. These calls do not require a save afterwards.
For a pipe, a new record within the intermediary table is created (in this case, a new
GroupUser record is created with the user and group relation). If the relation already
For a reverse lookup, the add method will set the reference instance to the calling record.
In this example, we have a Role model, a role reference on the User model, and a
ReverseLookup that goes through the User.role column. Calling the
role.users().add(john) would set the role column to the calling role instance on john,
and save the record. Again, this does not rqeuire saving afterwards.
Collection.create
The create method will act the same way as the add method, except instead of accepting
a record, it accepts a dictionary of column/value pairs and will dynamically create a record
vs. associate it.
Collections 48
ORB Framework
For this method to work, you will need to have defined what model is associated with a
collection. If the collection is returned as a part of a collector, then it will automatically be
associated.
In these examples, we first created a basic User record. The second example will create a
Group called 'reporters' , and then associate it with the user john. The final example will
create a new user automatically setting it's role to the 'Software Engineer' record.
This will modify 2 records and create 1 -- with a total of 2 queries that will be made to the
backend data store.
Deleting Records
Collection.delete
Bulk deletions can also be performed using collections. Again, depending on where the
collection comes from -- different deletion options will be available to you.
Collections 49
ORB Framework
Calling this will perform a delete on all the users whose name is 'John', in one go from the
database.
If you are working with the results of a collector (reverse lookup or pipe), you have a few
more options for removing records.
Collection.remove
In this example, removing the 'editors' group from the pipe will not delete the group
record. Rather, it will remove the record from the intermediary table. Both the user and group
will still exist at the end of the remove.
The second example, removing from a reverse lookup, will not actually remove either the
Role or the User record, but instead will set the User.role column value to null. This is how
records are found for reverse lookups, so removing is simply setting the value to an empty
value.
Collection.empty
This method is more similar to the Collection.remove method than the Collection.delete
method. What this will do is it will remove intermediary tables for pipes, disassociate reverse
lookups from their references, and do nothing for empty collections.
Collections 50
ORB Framework
In these two examples, the first will remove all GroupUser records where the user_id is 1,
this will not remove any groups or any users. The second will not actually remove any
records at all, instead it will update all User records whose role_id is 1, and set it to
None.
The next time the groups or users call is made for these records is performed, the
collection will be empty.
At this point, the users variable does not actually have any records in it. It just has the
knowledge that, when asked, it should go and fetch all User records and return them.
Depending on the action you take next, ORB will optimize how it interacts with your backend
store.
For instance, if all I care about is the number of users I have, and not the users themselves,
doing:
Will query the database for just the count of the users based on the selection criteria (in this
case, all records) and return that number.
But doing:
Will query the database for all the user records and cache the results. Then, when I call
len will not query the database at all -- it will determine that we've already loaded the
records, and will instead just ask for the length of the loaded list.
Collections 51
ORB Framework
It is important to understand what functions are available for collections, but it is also
important to keep in mind their relative expense.
Warning:
Collections 52
ORB Framework
Querying
One of the more powerful aspects of the ORB framework is it's querying syntax. This
language allows you to programmatically define complex queries for the system to process
later.
Structure
The structure of a Query is as follows:
The {column} value is the string name or field of a column, the {op} is the operator for the
query, and the {value} is the value to compare.
Where possible, we use standard logical operators for the query class, which is instantiated
with the Query class, commonly imported at Q .
Basic Operations
Querying 53
ORB Framework
Query Functions
If you need to dynamically modify the column value during the query comparison, you can
easily do that with ORB. These are functions that can be performed on a query object.
Query Math
Querying 54
ORB Framework
Similar to the query functions, you can also perform query math. This will do mathematical
operations to the column during the query:
>>> Q(column) + offset == value # add an offset to the column before comparing
>>> Q(column) - offset == value # subtract an offset to the column before comparing
>>> Q(column) * offset == value # multiply an offset to the column before comparing
>>> Q(column) / offset == value # divide an offset to the column before comparing
>>> Q(column) & check == value # perform a bitwise AND before comparing
>>> Q(column) | check == value # perform a bitwise OR before comparing
A common example scenario of when to use query math is checking for whether or not a flag
is set on an enumerated column:
Query Compounds
Queries can also be logically strung together to make more complex lookups. This is done
using the OR and AND logical operators.
Querying 55
ORB Framework
>>> # jane & doe => first_name is Jane and last_name is Doe
>>> User.select(where=jane & doe).values('username')
['jadoe']
You can invert a query using the ~ operator. This will do a mathematical inversion:
>>> # john & ~doe => first_name is John and last_name is not Doe
>>> User.select(where=john & ~doe).values('username')
['jhancock']
You can do multiple compounds together as well. Like math, parenthesis placement is
important, as that will drive what gets calculated first:
>>> # (john | jane) & doe => first_name is John or Jane, and last_name is Doe
>>> User.select(where=(john | jane) & doe).values('username')
['jdoe', 'jadoe']
>>> # john | (jane & doe) => first_name is John, or first_name is Jane and last_name is Doe
>>> User.select(where=john | (jane & doe)).values('username')
['jdoe', 'jhancock', 'jadoe']
Querying 56
ORB Framework
Warning:
You need to be careful with Python's & and | operators. They are processed first
within an equation, and can cause trouble when building a query. For instance: q =
Q('first_name') == 'John' & Q('last_name') == 'Doe' will actually raise an error,
because Python will interpret this as: q = Q('first_name') == ('John' & Q('last_name'))
== 'Doe' , which will be an invalid query.
If you are doing in-line queries, make sure to contain them within parentheses: q =
(Q('first_name') == 'John') & (Q('last_name') == 'Doe'))
Building Compounds
This flexible syntax provides a lot of power. However, it can be cumbersome for simple
comparisons. If you want to just build a query with a key/value pair, you can use the
Query.build classmethod:
Joining Models
Sometimes you will need to reference other tables within a query. For instance, let's consider
the following structure:
Querying 57
ORB Framework
import orb
class Login(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
password = orb.StringColumn()
profile = orb.ReferenceColumn(reference='Profile')
class Profile(orb.Table):
id = orb.IdColumn()
first_name = orb.StringColumn()
last_name = orb.StringColumn()
department = orb.StringColumn()
title = orb.StringColumn()
In the above example, we have a one-to-many relationship between Profile and Login --
where one user profile could have multiple logins associated with it.
Let's say we wanted to lookup all logins where their associated Profile's title is "Manager".
The profile column on the Login model is a reference to a Profile object. You could
query by any column of the Profile object ( profile.first_name , profile.last_name , etc.).
Querying 58
ORB Framework
Normally, the in_ method will convert the value to a list for the backend to compare. When
the value is actually a collection it can be optimized a bit more because we know how to
populate that collection on the backend.
Tip:
When using in_ or notIn on a collection of records, it is best to pass in the collection
itself so that ORB can optimize the lookup.
Model-specific queries are created by providing both the model and the column to lookup.
You can manually build joins in this fashion by using the output of one query to be the value
of another.
import orb
class User(orb.Table):
id = orb.IdColumn()
username = orb.StringColumn()
password = orb.StringColumn()
groups = orb.Pipe(through_path='GroupUser.user.group')
class GroupUser(orb.Table):
id = orb.IdColumn()
user = orb.ReferenceColumn(reference='User')
group = orb.ReferenceColumn(reference='Group')
class Group(orb.Table):
id = orb.IdColumn()
name = orb.StringColumn()
users = orb.Pipe(through_path='GroupUser.group.user')
Querying 59
ORB Framework
Now, if I want to lookup all users who are a member of the 'admin' or 'editor' group --
how would I do this?
The traversal here reads "select from User where the User 'id' equals the GroupUser 'user'
column. Then look for where the GroupUser 'group' column equals the Group 'id' column.
Finally, look for the Group 'name' if it is in the tuple."
This is a bit more complicated, but also more flexible when building complex queries.
But wait, why can't I just use a dot-notated shortcut? Actually, you can. We just wanted to
show you how to do it manually...
As you can see, you can also traverse collectors. In this case, a User's groups pipe will
already do the traversal through the intermediary table. From there you just need to filter on
the group's name property.
Querying 60
ORB Framework
Columns
Columns 61
ORB Framework
Databases
Databases 62
ORB Framework
Instead, it focuses on the core of web services -- leaving extensibility up to its users. We
have written a few libraries to help with integrating ORB based APIs with the Pyramid
framework and making ReSTful services out of them.
Installation
Note:
This integration will only work for Pyramid web servers. You will need to read through
the pyraid docs to get started with that server.
ORB support within Pyramid is done through the pyramid_orb middleware plugin, and is
built on top of the pyramid_restful project.
Pyramid 63