This is a long post. Don't worry, there are plenty of place to stop reading. You can stop at the end of each major section and have a complete picture for a given level of detail. Developers trying to see how to build a 4th party ecommerce application in KRL should read to the end to understand the complete picture.
The caption of Peter Steiner's legendary 1993 New Yorker cartoon reads: "On the Internet, nobody knows you're a dog." If you've paid attention to the Internet Identity Workshop logo, you know we use that concept for the conference, although our dog has a mask. But as a recent NY Times article, Upending Anonymity, These Days the Web Unmasks Everyone, points out, that's not really true anymore.
This erosion of anonymity is a product of pervasive social media services, cheap cellphone cameras, free photo and video Web hosts, and perhaps most important of all, a change in people's views about what ought to be public and what ought to be private. Experts say that Web sites like Facebook, which require real identities and encourage the sharing of photographs and videos, have hastened this change.
"Humans want nothing more than to connect, and the companies that are connecting us electronically want to know who's saying what, where," said Susan Crawford, a professor at the Benjamin N. Cardozo School of Law. "As a result, we're more known than ever before."
From Upending Anonymity, These Days the Web Unmasks Everyone - NYTimes.com
Referenced Wed Jun 22 2011 18:58:33 GMT-0600 (MDT)
This loss of anonymity is sometimes voluntary--say, when I post something on Twitter--but often is a side effect of our online activities. People call this "exhaust data." The point of this post is to illustrate an architecture for ecommerce that minimizes the tracable exhaust data from the activities associated with shopping for a product and trying to find the best deal.
The Scenario
One of the benefits of online shopping is the ability to quickly check out the same product from multiple vendors and find the best deal. There are lots of sites that try to do this for you, but they don't really take your individual situation into account and rarely give you comparisons between final prices (including discounts, taxes, and shipping).
In this demo, we're going to build a 4th party shopping assistant using some Kynetx rulesets. As I talked about in Building Fourth Party Apps with Kynetx last year, it is possible to imagine a Kynetx ruleset acting as a representative of the shopper (the 4th party) and other rulesets acting on behalf of the merchant (the 3rd parties). These rulesets can interact to provide the shopper with the information she needs while taking into account her personal situation without that personal data being shared with the merchant or their representatives in a way that is tracable back to the shopper.
In this demo, we'll build a simple offer app that shows the shopper what offers other merchants will make to them for the product they're currently viewing online. Think of this as a simple personal "request for quotes" (RFQ) system. We'll have two shoppers Sally, who lives in Utah and is an Amazon Prime member, and Bill, who lives in California and is in the military. To keep things simple, both shoppers have the same two favorite online merchants: Amazon and Home Depot. Further, they'll both start by looking at the same product, a Panasonic Microwave.
The scenario starts when the 4th party shopping app, we'll just call it the offer app, notices that the shopper is visiting a relevant product page and places a "want" button on the page. Here's what happens:
- Shopper visits Scott's Microwave Store
- Shopper clicks on the "want" button next to a product that has caught her idea
- Shopper gets personalized offers for the product from some of her favorite merchants
Here's what the offers look like:
Behind the Scenes
As far as it goes, that's nice, but the point here is to really build this, so let's get a little more detailed. The following schematic shows what going on behind the scenes.
There are only two Web events in the hierarchy, reflective of the simplicity of the overall user interaction: the pageview causes the "want" button to be placed when a place_want_button event is raised. When Sally clicks the "want" button, a product_found event is raised which sets off the rest of the interaction.
The product_found event is seen by the process_product rule in the offer ruleset. This rule eventually raises the rfq event. Merchant rulesets are watching for that event. The rfq event causes the discount rules to fire and eventually, the process_offer_solicitation rules wrap everything up. In this simple example, there's only one discounting rule in each merchant ruleset, but there could be as many as necessary.
When the process_offer_solicitation rules are done they raise finished_offer events that cause the process_offers and finalize rules to fire in the offer ruleset. The process_offers rule is looking for both offers to be returned, but more generally could look for the first two or three in a given period of time.
The pattern in the event hierarchy shows the overall structure of the system. The rules for the offer ruleset are at the top and bottom. The merchant rulesets operate, in parallel, in response to the rfq event and raise the finished_offer event. Their internal structure and operation is up to the merchant who owns them.
Now, let's look at the individual rulesets to see the detail of what's happening. We'll start with the scratch space module, move onto one of the merchant rulesets, and finish with the offer ruleset.
The Scratch Space Module
The various rulesets in the offer app need a way to share data with each other. They could attach it as attributes to the events, but that's cumbersome. Currently the Kinetic Rule Engine has no built-in means for apps to share data, so for purposes of this demo, I built a simple tag-space store as a scratch space that the apps could use. The scratch space is used to store information about the shopper and the offers that are returned.
The idea of a tag-space is that objects (in our case JSON encoded) are stored in a specific namespace and associated with one more tags. The objects can be retrieved by querying with tags. The module provides a function for querying the tag-space, getd that takes a namespace ID and a set of tags and returns an array (possibly empty) of objects that have been tagged with all of the submitted tags. The module also provides an action for storing new JSON objects called setd. Setd takes a namespace ID, a set of tags, and a JSON object to store.
We'll use the tag-space by picking a namespace for each shopper (in real life, we'd do this for each interaction to preserve anonymity) and storing relevant personal information such as preferred merchants, zipcode, and a list of discounts the shopper is entitled to in the tag-space. Merchant rulesets retrieve shopper information from the tag-space and store offers into it for the offer ruleset to use.
The Merchant Ruleset
For this demo, the Amazon and Home Depot rulesets are very similar, so I'll just go over the Amazon ruleset. In a real system, of course, that need not be the case. They would simply need to watch for the rfq event and raise a finished_offer event when they are finished, making appropriate entries in the scratch space along the way.
The Amazon ruleset contains a rule called prime_discount that is selected on an rfq event that determines the whether the shopper is a member of Amazon Prime (and thus qualified for free 2nd-day shipping) or not.
rule prime_membership {
select when explicit rfq
pre {
customer_id = event:param("customer_id");
discounts = tagspace:getd(customer_id, "discounts").head();
}
if(discounts.filter(function(x){x eq "AmazonPrime"}).length()>0)
then noop();
fired {
raise explicit event discount_offer with membership="Prime"
} else {
raise explicit event discount_offer with membership="Standard"
}
}
The prime_membership rule consults the scratch space for the customer with the tag discounts, tests the list of discounts to see if "AmazonPrime" is there, and then raises an event discount_offer with the membership event attribute set to either "Prime" or "Standard" depending on the result of the test.
I anticipate that rulesets will not do all the work, but will make use of other online systems that the merchant has in place. For purposes of this demo, I dummied up a system that takes information about the customer and product and returns an offer in JSON format. We declared the API as a datasource named offers in the global section of the ruleset:
global {
datasource offers <- "http://.../amazon.cgi";
}
The real work of creating the offer is done by the process_offer_solicitation rule which is selected when there has been an rfq event and a discount_offer event.
rule process_offer_solicitation {
select when explicit rfq
and explicit discount_offer
pre {
customer_id = event:param("customer_id");
zipcode = tagspace:getd(customer_id, "zipcode").head();
modelno = event:param("modelno");
oid = event:param("offer_id");
is_prime = event:param("membership") eq "Prime";
std_offer = datasource:offers({"modelno":modelno,
"zip": zipcode});
offer = {"price": std_offer.pick("$.price"),
"shipping": is_prime => 0.00
| std_offer.pick("$.shipping"),
"shipping_type": is_prime => "2nd Day"
| std_offer.pick("$.shipping_type"),
"tax" : std_offer.pick("$.tax"),
"notes" : is_prime => "assumes Amazon Prime member"
| "",
"url" : std_offer.pick("$.url")
};
tags = "offer|#{oid}|Amazon";
}
tagspace:setd(customer_id,
tags,
{"offer": offer,
"zip": zipcode,
"modelno":modelno,
"merchant": "Amazon",
"icon": "http://.../want/amazon_icon.png"
});
always {
raise explicit event finished_offer for a16x108
with merchant = "amazon"
}
}
After retrieving customer information from the scratch space, the rule gets a standard offer from the Amazon offer API (in the datasource:offers call) and then calculates a final offer using information from the discounting rule. The offer, along with other relevant information is stored in the scratch space using the setd action. Finally, the rule raises the finished_offer event to signal it is done.
The merchant rulesets shown here are simple, but the pattern is clear: the ruleset responds to an rfq event and raises a finished_offer event when it's done. In between, the ruleset can use as many rules and data sources as necessary to compute the merchants offer.
The Offer Ruleset
Normally, I'd skip explaining how the "want" button is placed on the page since that aspect is fairly rudimentary, but because we're processing Schema.org microdata about the product as part of that rule, it beats consideration. The offer ruleset makes use of a slightly modified version of Philip Jagenstedt's jQuery module for processing microdata.
rule place_want_button {
select when pageview "/want/"
pre {
want_button = <<
<span id="want_button"><img src="want%20button.png"/></span>
>>;
}
every {
after("#buy_button", want_button);
emit <<
$K("#want_button").click(function(){
var jsonText = $K.microdata.json(
"[itemtype='http://schema.org/Product']");
var prodprops = jsonText.items[0].properties;
var offer = prodprops.offers[0].properties;
var seller = offer.seller[0].properties;
app = KOBJ.get_application("a16x108");
app.raise_event("product_found",
{"prodname":prodprops.name[0],
"modelno":prodprops.model[0],
"produrl":prodprops.url[0],
"price":offer.price[0],
"shipping":offer.shipping[0],
"seller":JSON.stringify(seller.name[0], undefined, 2)
});
});
>>
}
}
Most of the work is done by JavaScript emitted by the rule. The after action places the button and the JavaScript places a listener on the browser "click" event that uses a callback function to process the microdata on the page and raise the product_found event to KRE.
The process_product rule is selected by the product_found event. The process_product rule is responsible for placing the notification on the page that the shopper will see with the correct structure for later rules to write into (the <div/> named offers). The notification also contains information about the product being quoted including the calculated final price for the product on the page the shopper is visiting.
rule process_product {
select when web product_found
pre {
price = event:param("price");
shipping = event:param("shipping");
final_price = price+shipping;
oid = "oid" + math:random(9999);
a = <<
<div class="prod">
<a href='#{event:param("produrl")}'>
#{event:param("prodname")}</a>
<br/>Model No: #{event:param("modelno")}
for $#{final_price} (including $#{shipping} shipping).<br/>
Sold by #{event:param("seller")}<br/>
<hr/>
<div id="offers">
</div>
</div>
>>;
}
notify("Compare Offers for #{event:param('who')}", a)
with sticky = true and width="300px";
fired {
set ent:oid oid;
raise explicit event rfq for merchants with
modelno = event:param("modelno") and
customer_id = customer_id and
offer_id = oid;
}
}
The rule postlude stores the offer ID (generated anew for each interaction) and then raises the rfq event with the model number, the customer ID, and the offer ID. As we saw, the merchant rules will use the customer ID as the namespace in the scratch pad and the offer ID to store data relevant to this offer.
The process_offers rule is designed to run when all the offers have been calculated by the merchant rulesets. The select statement in this demo is true when offers are complete from Home Depot and Amazon. In a production ruleset with many merchants, we might be content with the first two or three offers. The rule loops over the offers, calculates the final price for each merchant, and displays it in the notification placed on the page by the process_product rule above using the append action.
rule process_offers {
select when explicit finished_offer merchant "amazon"
and explicit finished_offer merchant "homedepot"
foreach tagspace:getd(customer_id,ent:oid) setting (of)
pre {
tax = of.pick("$..tax")>0 => "$"+of.pick("$..tax")+" tax"
| "no tax";
final_price = of.pick("$..price") +
of.pick("$..tax") + of.pick("$..shipping");
shipping =
of.pick("$..shipping") > 0 =>
"$" + of.pick("$..shipping") +
" for " + of.pick("$..shipping_type") +
" shipping"
| "no shipping fee";
prod_url = of.pick("$..url");
notes =
of.pick("$..notes") neq "" =>
"Notes: #{of.pick('$..notes')}"
| "";
merchant_id = of.pick("$.merchant").replace(re/ /g,"");
offer = <<
<div id='#{merchant_id}' class='offer'>
<a href="#{prod_url}" broder="0">
<img height="40px" border="0" align="left" src='#{of.pick("$.icon")}'/>
</a>
#{of.pick("$.merchant")} offers this product
for $#{final_price} (including #{tax} and #{shipping})
#{notes}
<a href="#{prod_url}" border="0">
<img src="green_arrow.png" border="0" valign="top" height="13px"/>
</a>
<br clear="both"/>
</div>
>>;
}
append("#offers", offer);
always {
mark ent:prices with {"price": final_price,
"merchant": merchant_id}
}
}
The finalize rule postlude stores a map with the price and merchant ID in an entity variable for use by the finalize rule which highlights the offer with the lowest price. The finalize rule is selected by the same eventex that was used to select the process_offers rule.
rule finalize {
select when explicit finished_offer merchant "amazon"
and explicit finished_offer merchant "homedepot"
pre {
lowest =
ent:prices.as("array").sort(
function(a,b){
a.pick("$.price")> b.pick("$.price")
}
).head();
lm_id = "#" + lowest.pick("$.merchant");
}
every {
emit <<
$K(lm_id).attr("style","background-color: palegoldenrod");
>>;
}
always {
clear ent:prices
}
}
The rule sorts the prices entity variable that was created by the process_offers rule to find the lowest price and uses the merchant ID to change the background color for that line in the display. Finally, the rule postlude clears the prices variable.
Conclusion
The system described here is a working demo of a system for creating offers in a way that protects customer identity from being divulged. The system uses a set of merchant rulesets, built and maintained by the merchants themselves, that create offers given data about the product and the customer along with an offer management ruleset that oversees the process for the customer--effectively acting as a 4th party.
The merchant rulesets are not single purpose, but rather can be written so as to be used by any number of systems that need offers on products for customers. The offer management ruleset used them to present an offer immediately to the customer. Another 4th party ruleset might use them to look for offers for things that the customer has placed on her wishlist.
By protecting customer's personally identifying information and presenting them with relevant offers, we create a system whereby shoppers feel safe in looking to merchants for information. At the same time, merchants can place highly relevant offers in front of people who have expressed an intent to buy.
