We noticed at Yahoo! that we were started to get lots of timeouts from Facebook’s APIs that we use on the Yahoo! homepage.

I’ve not heard anything official from Facebook yet, so this next part is all speculation.

I suspect that as a user updates their privacy settings, as forced to the first time they hit facebook.com now, Facebook flags that users account is dirty (something changed) and purges the cache of the users stream data.
Now that the cache is empty for a particular user, when a friend of that user views Facebook, the backend servers must go back to the master data source and confirm permissions on what is visible before it is displayed.

That is fine when you have a few users updating their permissions, if however you force ALL your users to do this, you site rapidly becomes overwhelmed, as you effectively removed your cache. This effect is known as “The Thundering Herd”.

As the load increases, due to all extra requests for data that would normally be handled by caches, you start to get race conditions as requests return and try to populate the cache. Also as the load increases, requests start to take longer as connections are kept open while backends are queried. Because connections are being kept open, the server stops handling requests, as there are no more connection slots and you start to get errors as clients can no longer connect.

How do you prevent this. Don’t push changes that require you whole userbase to update their records at once. Stage the release, starting with 1% until you can guage how much load it will generate on your servers. Then you can safely ramp up your change safely.

Tesco is starting the rollout of it’s new backend Grocery application, code named Martini (anytime, anyplace, anywhere).
They have opened up their new APIs from their grocery service to a few select developers. In fact very select, you had to attend to launch/briefing in person to register for the scheme.

A lot of time had obviously gone into preparing the event. With almost the whole conference room covered in brainstorm sheets and post it notes. These covered 5 main areas for potential developers to concentrate on or consider for their application. This was based on market and socioethnic research (hey that may not be the correct word but I didn’t write it down and I’m now on the train).

After an initial introduction, we were all invited to chow down on pizza, lager and wine (actually nice for change to get red and white wine available) and browse the brainstormings, only slightly hindered by Microsoft’s insistence that no one take food or drink into their meeting rooms.

The APIs look reasonably well thought out and the new Martini release adds some great new functionality, such as nutrition information and cheaper alternatives to products in your shopping basket.

There were some concerns voiced by some developers present at the briefing, quite rightly, over their security implementation, which requires the API consumer to forward the users login credentials, rather than implementing OAuth or another signature/token based authentication mechanism.
It looks like this will probably be addressed in a future release, once Martini is live and running Tesco’s existing shopping site.
Customer basket checkout has yet to) be finalised, due to financial and legal compliance.

Everyone who attended the event and wishes to participate in the affliate scheme will get signed up to TradeDoubler and will earn £5 for each new customer they manage to entice into signing up to Tesco’s online shopping service.
The tricky part of this is then the ongoing revenue.
For each basket purchased over the value of £50 the application developer gets a potential £0.10. Potential, as this is the proportional to the number of products added to basket by that particular application versus the total number of products in the basket.
Note also products in basket not items. 10 one penny chews would earn you the same as someone adding an iPhone or flatscreen tv. This is deliberate, to prevent skewing of apps towards higher value goods.

Having spent so long working on commercially sensitive applications, which is now in the open.

Anyway, apart from helping the Yahoo! OpenHack day, I also signed up to Tesco’s T-JAM event which certainly looks like it could be interesting.

Tesco’s API is a SOAP API, written in and for C# primarily, so it seems that some people had some issues getting started in PHP.

There is a revised API in the pipeline and also the addition of a REST API as well.

But here are some handy hints for getting started:

First off, PHP 5+ has a SOAP client (and server) built in, so here’s a quick script that generates a valid login to the TESCO dev API.

<?php
define('SOAPSERVER', 'http://www.lansley.com/TescoAPI/TescoAPI.svc?wsdl');
define('DEV_KEY', 'your dev key');
define('APP_KEY', 'your app key');

$user  = 'email address of test user';
$pass  = 'password of test user';
$tesco = new SoapClient(SOAPSERVER);

$params = array(
                'email'          => $user,
                'password'       => $pass,
                'developerKey'   => DEV_KEY,
                'applicationKey' => APP_KEY
              );
$sess = $tesco->__soapCall('Login', array('parameters' => $params));
print_r($sess);

Which generates the following output:

stdClass Object
(
    [LoginResult] => stdClass Object
        (
            [StatusCode] => 0
            [StatusInfo] =>
        )

    [session] => stdClass Object
        (
            [ApplicationKey] => your app key
            [BranchNumber] => 2055
            [Cookies] => stdClass Object
                (
                    [string] => Array
                        (
                            [0] => sessionTest=
                            [1] => v=WggAACp8E%3d%3d
                            [2] => CustomerId=9xxx9
                            [3] => CID=93
                            [4] => BTCCMS=h1qzwp%c%3d
                            [5] => UIMode=00000
                            [6] => PS=CSC=D
                            [7] => SSVars=CustomerName=Bob&basketTotal=&basketCount=&slotStart=&slotEnd=&addrName=
                            [8] => u=oQgttVAeDK%2bg%3d%3d
                            [9] => t=dgcAA28XbJWE%3d
                        )

                )

            [CustomerName] => Bob
            [DeveloperKey] => your dev key
        )

)

I’ve monkeyed around with the values in some of the keys (as these are my login credentials) but you can see the general structure. The session object needs to be stored and used in subsequent requests.

To interrogate the API and view the functions and types available, there are some PHP functions to help:

<?php
define('SOAPSERVER', 'http://www.lansley.com/TescoAPI/TescoAPI.svc?wsdl');
define('DEV_KEY', 'your dev key');
define('APP_KEY', 'your app key');

$tesco = new SoapClient(SOAPSERVER);

print_r($tesco->__getFunctions());

Which shows the following SOAP methods available:

Array
(
    [0] => LoginResponse Login(Login $parameters)
    [1] => LogEventResponse LogEvent(LogEvent $parameters)
    [2] => ProductSearchResponse ProductSearch(ProductSearch $parameters)
    [3] => ListProductOffersResponse ListProductOffers(ListProductOffers $parameters)
    [4] => ListProductCategoriesResponse ListProductCategories(ListProductCategories $parameters)
    [5] => ListProductsByCategoryResponse ListProductsByCategory(ListProductsByCategory $parameters)
    [6] => SetProductRatingResponse SetProductRating(SetProductRating $parameters)
    [7] => GetProductRatingResponse GetProductRating(GetProductRating $parameters)
    [8] => GetMultipleProductRatingsResponse GetMultipleProductRatings(GetMultipleProductRatings $parameters)
    [9] => ListBasketItemsResponse ListBasketItems(ListBasketItems $parameters)
    [10] => AddItemToBasketResponse AddItemToBasket(AddItemToBasket $parameters)
    [11] => ChangeItemQuantityResponse ChangeItemQuantity(ChangeItemQuantity $parameters)
    [12] => DeleteItemFromBasketResponse DeleteItemFromBasket(DeleteItemFromBasket $parameters)
)

And:

<?php
define('SOAPSERVER', 'http://www.lansley.com/TescoAPI/TescoAPI.svc?wsdl');
define('DEV_KEY', 'your dev key');
define('APP_KEY', 'your app key');

$tesco = new SoapClient(SOAPSERVER);

print_r($tesco->__getTypes());

Which shows the following types available:

Array
(
    [0] => struct ArrayOfstring {
 string string;
}
    [1] => struct ResponseStatus {
 int StatusCode;
 string StatusInfo;
}
    [2] => struct Session {
 string ApplicationKey;
 string BranchNumber;
 ArrayOfstring Cookies;
 string CustomerName;
 string DeveloperKey;
}
    [3] => struct ArrayOfProduct {
 Product Product;
}
    [4] => struct Product {
 string Id;
 string ImagePath;
 string Name;
 string OfferPromotion;
 string OfferValidity;
 double Price;
 int Quantity;
 float Rating;
}
    [5] => struct ArrayOfCategory {
 Category Category;
}
    [6] => struct Category {
 ArrayOfCategory Children;
 string Href;
 string Id;
 string Name;
 string RawHtml;
}
    [7] => struct ProductRating {
 string Id;
 int MyRating;
 float Rating;
 int Rating1Count;
 int Rating2Count;
 int Rating3Count;
 int Rating4Count;
 int Rating5Count;
}
    [8] => struct Login {
 string email;
 string password;
 string developerKey;
 string applicationKey;
}
    [9] => struct LoginResponse {
 ResponseStatus LoginResult;
 Session session;
}
    [10] => struct LogEvent {
 Session session;
 string eventText;
}
    [11] => struct LogEventResponse {
 ResponseStatus LogEventResult;
}
    [12] => int char
    [13] => duration duration
    [14] => string guid
    [15] => struct ProductSearch {
 Session session;
 string searchString;
 boolean getRatings;
}
    [16] => struct ProductSearchResponse {
 ResponseStatus ProductSearchResult;
 ArrayOfProduct products;
}
    [17] => struct ListProductOffers {
 Session session;
 boolean getRatings;
}
    [18] => struct ListProductOffersResponse {
 ResponseStatus ListProductOffersResult;
 ArrayOfProduct offers;
}
    [19] => struct ListProductCategories {
 Session session;
}
    [20] => struct ListProductCategoriesResponse {
 ResponseStatus ListProductCategoriesResult;
 ArrayOfCategory categories;
}
    [21] => struct ListProductsByCategory {
 Session session;
 Category shelf;
 boolean getRatings;
}
    [22] => struct ListProductsByCategoryResponse {
 ResponseStatus ListProductsByCategoryResult;
 ArrayOfProduct products;
}
    [23] => struct SetProductRating {
 Session session;
 Product product;
}
    [24] => struct SetProductRatingResponse {
 ResponseStatus SetProductRatingResult;
}
    [25] => struct GetProductRating {
 Session session;
 Product thisproduct;
}
    [26] => struct GetProductRatingResponse {
 ResponseStatus GetProductRatingResult;
 ProductRating productrating;
}
    [27] => struct GetMultipleProductRatings {
 Session session;
 string spaceDelimitedProductIDs;
}
    [28] => struct GetMultipleProductRatingsResponse {
 ResponseStatus GetMultipleProductRatingsResult;
 string productRatingsInfo;
}
    [29] => struct ListBasketItems {
 Session session;
}
    [30] => struct ListBasketItemsResponse {
 ResponseStatus ListBasketItemsResult;
 ArrayOfProduct products;
}
    [31] => struct AddItemToBasket {
 Session session;
 Product item;
}
    [32] => struct AddItemToBasketResponse {
 ResponseStatus AddItemToBasketResult;
}
    [33] => struct ChangeItemQuantity {
 Session session;
 Product item;
}
    [34] => struct ChangeItemQuantityResponse {
 ResponseStatus ChangeItemQuantityResult;
}
    [35] => struct DeleteItemFromBasket {
 Session session;
 Product item;
}
    [36] => struct DeleteItemFromBasketResponse {
 ResponseStatus DeleteItemFromBasketResult;
}
)

Hope this helps, not just with the Tesco Grocery API, but in general debugging SOAP calls from within PHP.