An overview of design values

As is often the case in the early stages of software projects, our goals for the Chandler Interpersonal Information Manager are ambitious, given our limited resources. But by choosing an architecture that uses the best technologies for what we're trying to achieve, and being smart about the development process, we should have a great leg up. We hope to streamline the development process via two main strategies: First, we'll build on as much existing open source software as possible, getting help from the open source community to evolve and expand it as necessary. Ideally, the vast majority of code in Chandler will be code we don't write. Second, we'll combine sound architectural decisions with improved development methodologies to make software a lot easier to write. Key to our success will be having a world -- class team of developers head up the effort.

As with most human endeavors, good software architectures evolve, and what is presented here is only an overview, or summary of our architecture values. As we get further along in development, we'll revise this document accordingly, including more details of the design.

Here's a summary of our primary architectural decisions:

• Build on open source software that supports open standards, choosing projects that are reliable, well documented, and widely used
• Use the Python language at the top level to orchestrate low level, higher performance code
• Design a platform that supports an extensible modular architecture
• For the desktop client, choose a cross platform U/I toolkit that provides native user experience
• Use a persistent object database
• Build in security from the ground up
• Build an architecture that supports sharing, communication, and collaboration

Guiding Principles

Use existing open source software that supports open standards, choosing projects that are reliable, well documented, and widely used

Much of our work involves figuring out which of the vast array of open source software is best suited for our task, then adding the code necessary to make it callable from Python. Some of this software doesn't currently meet our requirements for quality, performance, or feature sets, so we'll contribute toward improving projects we depend upon, as necessary. Hopefully these contributions to the open source community will make it easier for others to write quality application software in the future. The Mozilla project stands out as potentially having a number of key pieces we hope to reuse, the most obvious being the Gecko HTML layout engine and the HTML e-mail editor.

Use an interpreted high-level language to as the glue that holds together low level, high-performance pieces

Perhaps one of our more unconventional decisions is to use Python -- a dynamic, interpreted language. Compared to static languages like C++ or Java, dynamic languages tend to have poor performance and lack features for developing large programs like Chandler. But Python has many other features of modern programming languages, and its dynamic nature makes it easier to program than its static counterparts. To mitigate performance deficiencies, we plan to use Python only at the top level to orchestrate lower-level code written typically in C/C++. Much of this lower-level code already exists in open source form and in many cases is already accessible to the Python programmer. As for supporting large programs, Python is better equipped than most other dynamic languages and we hope to face only minor compromises that will be far outweighed by the advantages.

The Python standard library contains a vast array of modules for developing applications. Some of these are written completely in Python and others are written in C/C++. They include a number of important protocols like SMTP, POP, IMAP and HTTP, and a host of useful libraries, including an XML parser, socket libraries, a regular expression package, and much more.

Build a platform that supports modules at a variety of levels

By designing a platform with well-defined APIs, we hope to enable both ourselves and the open source community to easily write modules we call parcels that can be plugged in at a variety of levels. This will make it possible to replace portions of Chandler to make improvements or add functionality that we could never anticipate -- and conveniently share them with others users.

Examples of modules include the "viewer parcel," an area of the desktop user-interface which can view various kinds of items, for example, the calendar view for viewing calendar event items, an email view for viewing email items, a contacts view for viewing contact items, or a generic table view that can view items of any type. If you have some experience programming Python, we hope it would be relatively easy to add a new kind of data, for example your music collection including information about each song and write a music viewer that you could use to search and view your music.

Parcels can be used at a many different levels. They might include an API for filtering email that can be used for spam filtering, an API for plugging in new protocols used to enhance or improve communication, an API for extending the search capabilities in each view, or even an API to attach a script to a button in the user-interface.

Finally, we hope that these parcels and other building blocks like our persistent object database can be reused in unrelated software projects. This will make it easier to develop software and increase the quality and choice of available application software.

Use a cross-platform U/I toolkit that provides native user experience

One of our goals for the desktop client is to have a version that runs on Linux, Macintosh and Windows. We want each of these versions to be better, in terms of the user experience, than the existing counterpart on the same platform. This means the user interface must conform to each platform's application conventions. To implement native user interface elements on each platform, we could either use a different U/I toolkit for each platform, or try to find a single toolkit that allows implementing native elements for all three platforms. We chose the latter approach, and among the open source options, wxWindows/wxPython appears to be the best candidate.

There are limits, however, to the number of different user interface elements you can abstract to a single cross-platform U/I toolkit. For elements whose semantics aren't the same on all platforms, we'll have to write special-purpose code for each platform. Also, wxWindows has many limitations that will need to be addressed. In particular, the Macintosh port is new and still needs work, and on all three platforms, we'll need to improve and add support for various U/I widgets.

Choose data storage that's easy to use and evolve

Chandler has a repository, or database that stores a potentially huge volume of e-mails, attachments, contacts, documents, and so on, the structure of which will have to evolve over time. This database is probably our biggest architectural challenge.

If we design the interface to the database correctly, it should function equally well as a server or as a database local to the machine accessing it. One of the Chandler's design goals is to not require a server. However, if the server is available, Chandler could provide some extra features that rely on access to certain shared data always being available. It's not our intention, however, to provide the vast array of business features that server-based enterprise businesses require.

Improve and simplify the experience of sharing, communication, and collaboration

Now that we live in such a highly networked world, where many of our friends and family as well as workmates have a presence online, we can add several important new features that improve sharing, communication, and collaboration. For example, based on people's calendar entries, schedules can be displayed in a friendly way making it easy to see the if someone is available for an instant messaging, voice, or video conversation. A student planning a trip home for the holidays can overlay her calendar with her parent's to find the best time to schedule the visit. And when people update their contact information, the changes can be propagated to everyone who has subscribed to a copy of the contact.

At the highest level, items in repositories are accessible via their URL. However, the communication path varies depending on the availability of the client storing the data that needs to be accessed. By default, the best communication path will be chosen automatically by the system accessing the data. Each communication path, available via a common API, might have different capabilities for availability, latency, and capacity, and communicate using a variety of protocols. We use Jabber (XMPP) as a low-capacity transport to discover clients, which once identified, can attempt to establish a direct high-capacity connection such as SSL between themselves as peers. If a client is unavailable via SSL, the fallback is to XMPP, and if XMPP is not available Chandler will continue fall back and attempt to complete the communication transparently via email since data payloads can be included as MIME attachments in the email, but nothing would show up in your inbox.

Many people now have multiple computers: a desktop computer at work or school, a laptop, and a computer at home. Supporting work on a common set of data from these different computers requires either a server to share or store and forward data, or the ability to occasionally connect the different computers as peers to synchronize and replicate data that's changed since the last synchronization. We plan for peers to be able to modify data, whether connected to other peers or not. When they reconnect after being off-line from one another, an automatic mechanism will synchronize replicated data between each computer using the communication paths described above. A version and/or timestamp can identify which items need to be replicated, and on the rare occasion that multiple peers modify the same data, this versioning can identify which changes conflict so the user can choose the one to accept. When data resides on a server, or the peers can always communicate, such conflicts would not occur.

Another type of synchronization will allow users to make an "offer" of an item in their repository, such as their contact information, to which other users can subscribe. Changes to the item being offered can then automatically propagate to subscribers, ideally using the same underlying mechanism described above for synchronizing replicated data.

Use a Model View Controller architecture

The Model View Controller (MVC) architecture separates the back-end data and application (model) from the front-end user-interface (view). The controller links the view and model. One big advantage of MVC is that it makes it easier to add a new front-end, for example, a Web or PDA interface for Chandler, without modifying the model. MVC is also useful in large applications because the data model can be developed largely independent of the user interface code.

Security is hard

We're just beginning to work out our security strategy, and based on our preliminary exploration, it doesn't look like there's a completely transparent, 100 percent secure and easy-to-implement solution that interoperates with other clients, especially for the product without a server. Inevitably, we'll have to make some compromises. In any case, it has to be easy for users, otherwise they won't use it, and that would be worse than having no security at all.

For example, to ensure that only certain people or groups have access to data, we need to provide a mechanism for permissions or capabilities. At the implementation level we hope to provide a flexible fine-grained control over data access, but at the user level we plan to simplify access, in part, by associating items with a group of people and defining access for the group. We haven't yet worked out the details of implementation or even which model -- permissions, capabilities, or something else -- to use.

top