As we discussed in Personal Clouds as General Purpose Computers, when personal clouds begin to act as peers with other network services, people gain unprecedented power and leverage. Personal clouds can change how we related to everything in our lives, rearrange how we buy and sell products and services, and revolutionize how we communicate with each other.

For these changes to take place, personal clouds must be able to do more than store personal data and mediate interactions with it—as important as that is. Your personal cloud must run applications for you, under your direction.

When I say "run programs" you might be tempted to think of the kinds of applications that you run on your laptop or tablet: word processors, spreadsheets, and games. But it turns out there's already plenty of places to do that in the cloud and having those things run in your personal cloud isn't going to change much in your life.

Instead, I'm talking about programs that interact on your behalf with other online systems. Those programs end up looking more like services in an OS. For example, you might have an application that manages incoming notifications and forwards them to you in a channel you prefer based on context and content. Or you might have apps that intermediates transactions with online merchants. You'll have dozens, event hundreds of apps that help you manage and control your personal data.

I believe that the most natural programming model for a cloud OS (COS) is event-driven. Events indicate something happened—a state changed somewhere. Events often mean that an application should act. Events provide a powerful way to create a metaprotocol that can be used to define the various interaction scenarios that will be present in a personal cloud.

Events augment the traditional request-response programming model of the Web with one that drives action independent of the user. This is a critical component of any system that promises to upset the power structure of the client-server model. David Siegel eloquently explains why:

Adaptability is event driven. It's very different from the demand-driven systems we have today. If something happens in front of you, whether you're on a bike path, driving down the freeway, or flying at 30,000 feet, the system (all participants and their equipment) adjusts. When you take a pill, don't take a pill, hit a golf ball, reschedule an appointment, get in your car, or walk near a store that has something on your shopping list, the event triggers a response and keeps other people up to date automatically. In an event-driven world, we don't know which apps we need, and it won't matter. A piece of code sitting in the cloud that is perhaps almost never used is nevertheless ready to respond to something unusual, and we may only learn about this software service after we needed it. An event-driven world is designed to change as the data changes.

From Apple and the Cloud: A Cautionary Tale | Xconomy
Referenced Mon Apr 09 2012 14:58:15 GMT-0600 (MDT)

As David points out, event-driven architectures are adaptable in ways that demand-driven architectures aren't. Making events work in the cloud on behalf of people requires and event-based programming model specially designed for that task.

Personal Event Networks

The event-based programming model for the COS is called a personal event network (PEN). Personal because each person—entity, really—has their own. A network because there is an interconnected collection of programs interacting in the PEN via event-based protocols.

The basic unit of execution in the PEN is a rule. Rules connect events to actions. Applications in the PEN are collections of rules, or rulesets.

The primary duty of the personal event network is scheduling rule execution based on incoming events. The PEN does this by calculating an event salience tree for the PEN based on the rulesets that the owner has activated. The event salience tree allows the PEN to quickly determine which rules to execute for any given incoming event.

The PEN listens for events on event channels. A given PEN can have a virtually unlimited number of event channels, allowing them to be assigned individually to event generators. This gives the owner of the PEN complete control of inbound events. If an event generator isn't behaving as the owner would like, the event channel can be shut down without affecting any of the other relationships that the PEN has.

The following diagram shows events coming into a PEN on various event channels:

You can see that events might be coming from many different devices and services. Only a few apps are shown. The Twilio service might be raising events around incoming phone calls or SMS messages. REI, Visa, and the flowershop might be raising events about purchases or outstanding orders, you car might be raising events about needed maintenance or its location, and so on.

While the preceding diagram only shows three, there might be hundreds of rulesets installed in the PEN watching for various events. Multiple rules can respond to the same event and rulesets needn't necessarily coordinate their responses to a given event. The architecture lends itself to loosely couple collections of rules.

KRL

The PEN sets up a programming model based on events. Certainly we could respond to events in any number of ways, but we have created a programming language, KRL, or the Kinetic Rule Language, specifically for building applications in the PEN.

KRL has several features that are carefully designed to make writing applications in the PEN easier:

• Rule-based programming model—Rules are a natural way to respond to events. Rules in KRL are structured using the event-condition-action pattern. That is, rules link events to actions. When a particular event occurs, if certain conditions are true, then take an action. The event and action are fairly obvious components, but don't overlook the condition. Conditions allow rules to take into account context, rather than simply responding to what happened. Rules represent composable chunks of functionality.

• Event expressions—Rules are not limited to responding to only simple events. KRL contains an event expression language that allows developers to specify complex event scenarios. For example consider the following KRL event expression:

  select when web pageview 
                "/support/(\\d+)" setting(issue_number)
       before email received 
                subj.match(re/issue_number/) 
  

  This event expression indicates that the rule should be selected when a particular Web page is viewed before an email is received. The expression can be thought of as filtering two event streams, those for Web pages being viewed and those for emails being received.

  But there's more to this expression than a simple temporal relation between two event streams. Notice that the issue_number is being set for the web:pageview event based on the page's URL and tested when the email event is received. This event expression is doing the equivalent of an SQL join across the event streams, only matching when the web:pageview event and the email:received event are related in a specific way. KRL event expressions are like SQL for the events in a PEN.

• Cloud-based identity context—Each PEN has an independent identity and the rulesets that operate within it operate specifically for that identity. For example, KRL has variables with values that persist from invocation to invocation of a given ruleset. These persistent variables are entity specific—that is, the values stored are for the entity who owns the PEN. This is quite powerful since it means that most applications don't need an external data store. But more importantly, we can use this entity-specific persistence to maintain online context for the owner of the PEN including links to external data or services.

• APIs look like libraries—KRL is designed to ease the burden of working with Web-based APIs and their attendant protocols (e.g. OAuth). APIs form the libraries of the cloud OS. KRL makes it easy to bind multiple APIs together in a ruleset. KRL provides primitives for handling the predominant serialization standards like JSON, XML, and RSS, as well as providing modules for popular APIs and the ability for programmers to create and share modules for other APIs.

At present the only way to create programs in a PEN is using KRL. That needn't be the case. In the future, other programming languages might be used. They would need frameworks to understand events and the PEN's persistence model. For now, however, it's best to think of KRL as the C of the PEN; KRL is how personal event networks are programmed.

Services

Just as a computer operating system provides common services—like printing—that any program running on the OS can use, a cloud OS runs services for rulesets running in the personal event network. The first of these services we have developed is for notifications.

We have defined a draft Notification Event protocol to show how notifications messages will be handled in a personal event network. The standard allows developers to write notification services. PENs will include a notification service by default, but the user can replace or augment the default service handler by adding rulesets that respond to notification events. In effect, the Notification Event protocol is the printer interface for a personal event network.

With a notification service in the PEN, KRL developers don't have to build notifications into their rulesets or worry about meeting the user's demands with regard to how they want to be notified. This makes writing rulesets easier because common operations and services can be handled by the COS.

In addition to notifications, we anticipate common services around such things as contacts, calendars, and to-do lists, and other key aspects of a user's personal data.

Unprecedented Power

Event-driven programming models are powerful because of several important, fundamental aspects:

• Receiver-Driven Flow Control—Once an event generator sends an event notification, its role in determining what happens to that event is over. Downstream event processors may ignore the event, may handle it as appropriate for their domain, or may propagate it to other processors. A single event can induce multiple downstream activities as it and its effects propagate through the event-processing network.

  Unlike demand-driven interactions, event notifications do not include specific processing instructions. For example, if my phone receives a call, it can send the phone:inbound_call() event and the stereo system can interpret that as "turn down the volume". In a demand-driven architecture, my phone would send the turn_down_volume() instruction to the stereo instead. We refer to this property as semantic encapsulation because processors encapsulate the semantics of the event processing they perform and event generators are largely ignorant of those semantics.

• Higher Decoupling—Compared with other system architecture styles, event-processing systems exhibit higher decoupling along several important axes:

  1. Because event notifications do not contain processing instructions, destination information, and other details about how an event should be processed, the schema of the event is simple and flexible allowing less coordination between event generator and event processor. This makes events a convenient and legitimate means of creating lightweight protocols.
  2. Event generators do not necessarily need to know what processors are interested in the event. The event generator sends notification of the event to the event channel and, consequently, the associated event network, not to a specific processor.
  3. System components can be added or removed with less coordination in the overall system. Other components that want to respond in their own way to an phone:inbound_call() event can join the network without the event generator or any existing event handlers being affected. Similarly, components can be removed without the event generator and other event handlers being updated. This allows functionality to be layered.

• Near Real-Time Propagation—Event processing systems work in real-time in contrast with batch-oriented, fixed-schedule architectures. Events propagate through the network of event processors soon after they happen and can further affect how those processors will interpret and react to future events from the same or different event generators. Personal event networks thus allow a more natural way to design and create real-time information systems. As more and more information online gains a real-time component, this kind of processing becomes more and more important.

Personal clouds, running a cloud OS, with location-independent, semantically correct access to personal data from around the Web, and running applications that interact with other online services for the owner's benefit promise to usher in a new Web of unprecedented power and convenience. This isn't possible in systems that merely respond to user interaction from a browser or an app as is done in current client-server systems. For this future to occur, people need access to systems that operate 24/7 and see things—events—as they happen. That's the power of the event-based programming model embodied in personal event networks and made possible with KRL.

The next blog post will round out this series by showing how personal clouds communicate and discuss cloud federation through event subscription.