One of the technologies that seems to be picking up steam lately is the PDS or personal data service. The PDS goes by other names as well. David Siegel calls it a "personal data locker" in his book Pull. Drummond Reed has determined the right name is personal data service rather than "store." I talked about the problem with the name and enumerated some principles for personal data services after IIW DC. I'm certain that the PDS will be a significant topic of conversation at the upcoming Internet Identity Workshop XI in Mountain View, CA (register here).

Why all the fuss? I believe it's because the PDS is the centerpiece in a new kind of Internet; one where the individual sees significant increased utility from the use of their personal data in their behalf instead of having it used against them. At the same time, the PDS offers increased privacy over the current regime.

That said, I think a PDS might be more of a pain than a help if a PDS doesn't come with accompanying automation. If it's just one more thing to manage, then I don't need it. On the other hand, if the PDS helps me by making mundane tasks less onerous and focuses my attention on the decisions that I really must make, then it's a win.

Of course, if you've been following along, you've probably already guessed that I'm thinking the Kynetx Rule Language, or KRL, is the perfect way to automate PDS tasks.

So, without further ado, here's a five minute video that shows a conceptual demonstration of how a PDS and KRL can work together. This conceptual demo illustrates the opportunities that are available for automating the contextual activities that people undertake every day. At the heart of the demo is a personal data store and Kynetx. The interactions are all done using real Kynetx applications that are plumbed in a realistic manner. The scenario uses 5 different APIs and a dozen individual rulesets in the Kynetx system. In the scenario, Scott Phillips gets bad news from his radiologist: he needs surgery. You'll see that a personal data store and a collection of loosely coupled Kynetx apps automate the frustratingly disjointed activities associated with Scott's bad news and focused his attention so he can complete the tasks with the least amount of effort.

Project Neck Pain

I commissioned this demo in mid August in preparation for Doc Searls' VRM+CRM workshop on August 25, 2010. We called if Project Neck Pain (PNP) and it involved much of the company. The goals were

1. Create a compelling demo of what Kynetx could to to make a PDS useful and worthwhile
2. Produce a large application comprising multiple cooperating rulesets
3. Find out where the Kynetx Network Service lacked features in support of goals (1) and (2)

I couldn't have been happier with the result as it achieve all three goals. The video is proof of (1). I'll talk about (2) below. We extended and polished the platform in some significant ways in support of (3). My recent post on building event intermediaries is one small piece of that.

In what follows, I'll describe the various pieces that went into PNP.

Endpoints

To understand what happened behind the scenes in this video, you need to understand a little of the architecture of Kynetx. In the Kynetx Network Service, or KNS, active clients, called "endpoints," raise events. Rules inside the Kynetx Rules Engine respond to those events to cause an action. When an event is raised one or more rulesets might respond to that event on the user's behalf. In the case of the demo, there were four different endpoints involved:

• a web endpoint in the form of a browser extension,
• an email endpoint that uses IMAP to watch a mailbox,
• a telephony endpoint that uses Twilio to make phone calls and respond to user input, and
• the PDS itself.

The idea that a personal data service be not simply a repository of personal data, but an active participant in coordinating activities in behalf of the user is a key piece of the Kynetx vision for how personal data will transform what we think the Internet is and how it works. In the demo, the PDS is a key actor that not only responds to API requests, but also raises events.

As outlined in our free white paper: The Kynetx Rule Language - The First Internet Application Platform (PDF), there is a vital link between events raised by endpoints and rules. For example, the following rule fires off the phone confirmation when the PDS signals that the appointment has been chosen:

rule start_confirm is active {
  select when kpds update_event key "chosenappt" 
           or kpds create_event key "chosenappt"
  pre {
    phoneNum = (datasource:pds({"key":"phone"}))
                            .pick("$.value.number");
    appt = datasource:pds({"key":"chosenappt"})
                            .pick("$.value");
    confirmed = appt.pick("$.confirmed");
  }
  if((confirmed neq "") && (confirmed == 0)) then {
    http:post("https://secrets:here@api.twilio.com/Calls.json") 
     with params = 
      {"Called":phoneNum,"Caller":"8018954878",
       "Url":"http://webhooks.kynetx.com/h/a8x54/outboundcnct"};
    send_directive("log") with 
      app = "PhoneConfirm" and
      desc = "Confirmation Call Started";
  }
}

This rule is selected when the PDS signals that the appointment has been chosen. Note that there are two different ways that might happen and the select statement takes both into account. It gets Scott Phillips' phone number, the appointment data, and doctor information from the PDS in the rule prelude (pre) and then initiates the phone call with Twilio using an HTTP POST if specific conditions are met. One key feature of the event model is that the PDS doesn't have any idea what will happen when it signals that the appointment is set. The PDS simply raises the event. What happens depends on what apps the user has installed and what rules are in those apps.

APIs

In addition to the various endpoints involved in the demo, it also uses multiple APIs, including the follow:

• PDS - the PDS has an API as you can see in the preceding rule. Not only does the PDS allow data to be queried in a permissioned way (right now using OAuth), but it also allows values to be updated or created. At present our prototype PDS doesn't have a generalized schema beyond what is needed for this demo.
• Twilio - as already noted above, the Twilio API is used to make phone calls and gather responses from the user.
• Google Calendar - we are readying Scott Phillips' schedule from his Google calendar using their API.
• Flickr - the pictures painted on Scott Phillips' dashboard are coming from his FLickr account via the API.
• Weather - the weather data is coming from the Yahoo! weather API

The ability to freely use APIs makes the demo very powerful because of the data that can be pulled in. If the demo were real, there would likely be a dozen more APIs that would be useful in helping choose a doctor and schedule an appointment.

Rulesets

Our vision entails multiple coordinating rulesets cooperating in a loosely coupled manner to help the user. Events allow the system to be loosely coupled because, as mentioned above, the raiser of an event does not know who may be listening or what action might be taken because an event is raised. Multiple rulesets may be listening to and event without being aware of each other.

Our goal in creating the demo was to create apps that we thought might actually exist. We wanted apps that had a good backstory, even if they were demo apps. I'll describe the primary rulesets in this demo and how we envision they may really exist.

• Dashboard - the dashboard in the demo is painted by a ruleset. There's no real web page to speak of. Just a blank page with a few divs for structure. Everything on it from the banner, to the weather and time, are painted on the page by a Kynetx ruleset. This app might be provided by the PDS manufacturer.
• TODO List - the TODO list and its management are a separate app that might be provided by the PDS manufacturer or be a standalone component that the user installs according to their own preference.
• PDS Activity Stream - again this is a standard component for the PDS that shows detailed logging data
• Flickr - reads the Flickr feed of the user and shows pictures. This might come with the dashboard or be something the user has found and installed to work with the dashboard.
• Healthcare Action Items - watches the user's email for messages from the health care provider and adds relevant TODO items to the TODO list in the PDS. You can imagine that the radiologist is using a patient management system (PMS) to communicate with the user and has provided a PDS app to the user that is watching specifically for messages from the PMS. The app might be doing other things as well such as proactively reminding users of medical alerts, looking for appointment reminders, and so on. In the demo, we just looked for text patterns, but RDFa or microformats in a HTML message would provide more opportunity to grab meaningful, structured data from the email.
• Doctor Choices - responds to the TODO item about choosing a surgeon by showing the user some choices and annotating the doctors who are in the user's insurance network. This app might come from the user's health care network or an independent provider.
• Select Doctor - overlays the doctor's professional "about" pages with relevant data, augmenting what's there with recommendations from the user's social network, showing data from other Web sites, and any relevant ratings and reviews. In our scenario, we've imagined this app came from Angie's List or a similar service provider ranking site.
• Set Appointment - compares the user's calendar (Google in this case) with the doctor's free-busy schedule so find the intersection and display three choices for the appointment. The app also sets the appointment in the user's calendar and transmits it to the doctor. Currently we know of no standard way to discover calendar data for a service provider. Such a capability would be critical to an app like this working seamlessly.
• Phone Confirmation - called the user with relevant appointment data and asked for confirmation. Note that this app was calling on behalf of the personal data store. This app might be part of the appointment scheduling system or a more general "confirmation" app that the user has installed to confirm certain changes to PDS data using a secondary channel.
• Medical Information Transfer - responds to requests from medical providers for information from the PDS. This might be specific medical information or part of a more general information request system that requests and records the user's authorizations and data transfer preferences.
• Global Configuration - while it wasn't part of the demo that you saw, there's also a separate app that is used to configure the PDS and reset certain fields for demo purposes.

This set of eleven rulesets cooperate to deliver the experience that we saw in the demo. They are independently using the PDS and working with whatever APIs are relevant to their function. The PDS is the centerpiece of the demo since all the apps and functionality revolve around it. The individual rulesets know nothing of each other for the most part. They just do their piece and use events to message other components.

Events enable loose coupling, but they're not magic. While we've avoided it through design in this demo, loosely coupled apps may interfere with each other and give non deterministic performance in real life. For example, two apps may need to write the same field in the PDS. The PDS will need to incorporate appropriate data isolation techniques and, where ordering is important, apps will need to find ways to serialize themselves. Events can do this, but it's not automatic.

Life Events: Helping Users Achieve Purpose

Back in 2001, when we were envisioning how egovernment would work at the State of Utah, we came up with a concept called "life events." The idea was that most people interacted with government when something happened in their life that compelled them to. Further, those events almost never cleaved along departmental lines. A typical life event, like "moving to Utah" involved multiple interactions with many different government and non-government entities at all sorts of levels.

At the time, we envisioned that we'd build out tools on the Utah.gov portal that would help constituents walk through the various steps. The only one that ever got built out, as far as I know, is the one-stop business registration application. I've used it and it works surprisingly well. The reason that more didn't get built was that a server-side portal is just the wrong technology for coordinating all this.

One reason why a server is the wrong place to do this is that the server is missing all the help that active clients provide--specifically the ability to raise relevant events and respond to directives. As a consequence, the server has to simulate this by requiring the user to enter data, click buttons, and select things. That is, a server turns the user into the active client.

Another reason the server is wrong for this is that it's too tightly coupled. In a way, the whole Web services play was a complicated way of trying to create interactions between APIs on the server side. You could make this all work with Web services but it would be quite brittle. Of course, there are Web services technologies for creating more loosely coupled systems, but they're still working on the server away from the user.

In 2010, I think we're a lot closer to actually being able to help people with the tasks and activities that matter to them--their purpose for being online at any given time. The demo shown above is an example of this.

The answer isn't a big, one-size-fits-all, server-side, portal-based application, but a set of loosely-coupled, cooperating apps coordinating around a personal data service. The overall experience is infinitely customizable because any of the components can be traded out by the user. Further, the experience is incremental, meaning that I don't need all of the apps. I see incremental benefit as I incrementally install apps. The experience grows and changes as I customize my environment to suit my needs. What's more, changing out an app for another one doesn't require upgrading or changing the other apps.

Conclusion

There are several important ideas for me in Project Neck Pain that bear emphasis

• The experience focuses the user's attention on the things that matter, critical decisions, rather than making the user fuss with instigating every interaction and manage the details.
• There was no Web site involved in delivering this experiece except for a very simple Web page served with a few divs for structural purposes. Everything else was painted on the page using a Kynetx app.
• You can write event-driven applications that knit together multiple domains like Web, email, and phone. This is a powerful idea that is difficult without the concepts of an active client and events.
• Developers program loosely coupled applications that are event-driven. KNS with a PDS gives developers a very flexible platform for building all or just pieces of any given experience.
• KRL provides significant help in creating apps that use multiple APIs and interact using events. KRL was designed from the ground up with the idea of making writing such apps easier. Having a domain specific language adds significant mental leverage.
• You can build large applications in Kynetx using multiple cooperating rulesets. We've further tested this with another large app that we built for a customer in the last few weeks.
• A PDS without an accompanying automation system is going to be a pain in the neck. A PDS will be just one more huge, complicated thing to manage if it's not a participant in an app ecosystem.
• The demo shown in the video isn't really about health care, but rather, more abstractly, about picking a service provider and scheduling the service.

If any of this is interesting to you, I invite you to signup for a free Kynetx developer account and start playing around. We're happy to help--remotely or in person--and will gladly share the techniques and even the code we used to create this demo.