If you've followed my posts on the Fuse Architecture, you'll likely recognize this picture:

The architecture uses a pico to represent each vehicle, as well as one for the owner and one for the fleet. This organization provides essential communication pathways and establishes a structural representation of ownership and control.

I've also shown architectures where there was more than one owner, not to mention having relationships with everything from manufacturers, service vendors, and potholes.

With all that promise, however, I built Fuse with support for just one owner. That was a reasonable simplifying assumption when I was just getting started, but last week, with some prodding from Jim Pasquale, I decided I'd explore what it would take to support multiple owners, like so:

Introductions

The first step was to create the services inside the owner and fleet picos necessary to support introductions and subscriptions. Each pico subscription is unique. Each owner has a separate channel to the fleet and those channels have unique names.

I determined that the best way to support this was to re-use existing inter-pico subscription functionality as much as possible since that have been proven to work reliably over several years. The introduction process goes something like this:

1. The prospective owner asks the current owner for an introduction to the fleet
2. The current owner asks the fleet for a new channel and name to give to the prospective owner
3. The prospective owner uses the new channel and name to subscribe to the fleet

The diagram below shows how this works. Green interactions depend on existing CloudOS services. Blue arrows are new interactions I wrote specifically to support introductions. I didn't implement the black interactions for this experiment, but they'd be needed to roll this out for Fuse generally.

The fleet will only give up introductions to a pico with which it has an existing FleetOwner relationship. The fleet will only honor the subscription request if it is one that it has created and has the right name and channel identifier (shared key).

The overall experience for Fuse owners could mirror something like the Forever experiment we did several years ago. We'd likely build it into the Fuse Management Console (FMC).

The introduction pattern shows up in multiple place, albeit with different channel relationships (beside FleetOwner). For example, a similar process could be used in transferring ownership of a vehicle between two fleets. The primary difference would be that the fleet would drop the subscription to the old fleet once the transfer was complete.

Using Backchannels

I knew that giving a fleet two owners would cause some problems. After all, none of my testing—not even my mindset while programming—took two owners into account. Initial testing showed that both owners of the fleet could log into FMC and manipulate the fleet. So far so good.

The first place a problem showed up was weekly reports. Fuse sends each owner a weekly report. This can be disabled in the preferences. When enabled, the owner's pico has a scheduled event that fires once a week to send the report. The owner doesn't know how to generate a report, so it tells the fleet to generate a report. The fleet pico generates the report asynchronously.

The fleet, however, doesn't know how to email the owner. So, the fleet pico routes the completed report back to the owner for mailing. That's where things went wrong. Here the rule that routes events back to the owner when needed:n

rule route_to_owner {
  select when fuse email_for_owner ...
  pre {
    owner = CloudOS:subscriptionList(common:namespace(),"FleetOwner")
              .head()
              .pick("$.eventChannel");
  }
  event:send({"cid": owner}, "fuse", event:type())
     with attrs = event:attrs();
}

The problem is the head() operator. It simply returns the first member of the subscription list.

You can imagine what happened. When reports got sent out, one owner (which ever one happened to be first in the subscription list) received two reports and the other owner received none. This was easily remedied by having the rule find the owner channel based on who asked for the report in the first place (determined by the incoming channel as returned bymeta:eci()):

rule route_to_owner {
  select when fuse email_for_owner ...
  pre {
    owner_subs = CloudOS:subscriptionList(common:namespace(),"FleetOwner");
    matching_owner = owner_subs
                       .filter(function(sub){ 
                                 sub{"backChannel"} eq meta:eci() 
                               });
    owner = matching_owner.head().pick("$.eventChannel");
  }
  event:send({"cid": owner}, "fuse", event:type())
    with attrs = event:attrs();
}

This solves the problem by using the backchannel to find who sent the initial event. Other times, it might be best to lookup the owner by pico name or other data. This is a good pattern for event routing in general: don't depend on stored or specific channels for communicating with other picos, look them up instead. In general this is a good idea since event channel identifiers can change.

I'm sure I'll find a few more problems as we play with this some more, but it was nice to know that the existing subscription process could form a large part of the introduction process and that finding the right owner involved only minor changes to look up, rather than rely on specific, channels. One of the things I like about modeling with picos is that having unique, persistent objects to represent real-world entities and concepts (like owners, fleets, and vehicles) makes handling concepts like multiple owners straightforward.