Cut from the North

Handling “Session key invalid or no longer valid” errors from Facebook

kettle — Thu, 10 Sep 2009 12:55:33 +0000

I’m not sure if it’s supposed to work this way, but Facebook’s PHP client library method get_loggedin_user() doesn’t do the best job of keeping track of valid sessions. I seem to recall that even their demo FB Connect application experienced this error… It’s easy to reproduce on many FB Connect sites; log into a FB Connect site, then go to Facebook, logout of Facebook, and return to your FB Connect site. Reload the page and you’ll likely see a Session key invalid or no longer valid error followed by a trace.

So simple solution would be to catch the error; problem is that I don’t think it’s terribly obvious how to properly release the session if an error is caught… Using CakePHP, I added the following to my beforeFilter method:

if($this->facebook->get_loggedin_user()):
    try {
        $this->user = $this->facebook->api_client->fql_query('SELECT uid, pic_square, first_name FROM user WHERE uid = ' . $this->facebook->get_loggedin_user());
    } catch (Exception $ex) {
        $this->facebook->clear_cookie_state();
    }
endif;

So the only way to reliably test whether or not you have a valid session (since get_loggedin_user() can’t be trusted) is to run a method that requires one, and see if it throws an error. Sure, it adds overhead which is why I’d recommend a light FQL query which will return data I need anyway, but it’s much better than throwing fatal errors.

If an error is found, clear_cookie_state() seems to effectively destroy the session, and get_loggedin_user() will properly return false in all subsequent code.

Twitter’s API returning HTML instead of XML / JSON

kettle — Wed, 09 Sep 2009 21:38:12 +0000

I wasn’t going to write about this, but it’s been two weeks now, and more than a few people are agonizing over their Twitter API based apps not working. In case the title didn’t give it away, Twitter has been intermittently returning an empty HTML page as a response to API calls. I’m making XML requests, but the issue is also plaguing JSON requests. More about it here…

In an ideal world, a RESTful response should always be in the format requested. From what I’ve read, it appears this issue may be caused by recent measures to help prevent DoS attacks. Either way, I couldn’t wait any longer for Twitter to address the issue. Hopefully this helps someone else out…

My issues began when I first started building an application based on the twitter datasource. My Twitter data iterators kept throwing errors. By debugging the response before letting the datasource convert it into an array, I saw the request was retuning a 200 success header, and the following HTML:

Based on other user’s experience, it seemed waiting about .3 of a second usually cleared up the HTML response issue, and within 2-3 additional requests, you retrieved the data you were after. So I modified my _twitterReqest method, a simple wrapper for my calls to the datasource, to make the same request up to 5 times with a .3 second pause between requests until the data was returned in an acceptable format.

If 5 times wasn’t enough, I output a message explaining to the user that the API is unresponsive at the moment. So far, this message has yet to appear, and looking at my logs, I’ve never had to retry a request more than 2 times.

As soon as I get the data I want, I cache it for 24 hours, hopefully avoiding further issues for another day.

Here’s a quick look at my method:

function _twitterRequest($method = null, $params = null, $tries = 5, $delay = 300000){
    /*
     * This wrapper method is necessary b/c of issue documented here:
     * http://code.google.com/p/twitter-api/issues/detail?id=968
     * http://code.google.com/p/twitter-api/issues/detail?id=1014
     *
     * iterates through api calls with a .3 second delay to allow twitter responses to clear
     * has yet to fail
     */
    if($method):
        $try = 1;
        while($try <= $tries){
            $response = $this->Twitter->$method($params);
            if(isset($response['Hash']['error']) || isset($response['User']) || isset($response['Users'])){
                return $response;
            } else {
                $this->log($method . ': retry # ' . $try);
                usleep($delay);
                $try ++;
            }
        }
        return false;
    endif;
}

It’s frustrating to have to test against 1 + 1 equaling 2, but I guess it’s just the price of developing with 3rd party platforms… The moral here is always check your data against the data format you’re expecting. Don’t assume anything!

Trim down Twitter API responses with CakePHP’s Set::extract

kettle — Wed, 09 Sep 2009 12:56:08 +0000

I’ve been working a lot with the Twitter API lately; it’s been a frustrating experience, especially after being spoiled by the substantially more robust Facebook platform. One issue of particular concern was the volume of data being returned per request.

I am caching the response data, and I found that retrieving a list of 100 followers was storing over 150KB of cached data. One popular Twitter user could then easily be accounting for well over 1MB of cache files. I only needed about 4 fields from the XML response. I also knew the cakePHP Set class was the answer; I just didn’t know which method. So after a lot of trial and error, I figured out how to cut the fat with Set::extract. I’m not sure if this is the most efficient solution; also not sure if my regular expression is efficient… never did take the time to learn regular expressions. But my cache files are now ~18k vs. the ~150k they used to be.

I’m using the Twitter datasource which transforms Twitter’s XML response into an array. This is necessary as Set::extract expects an array, not an XML object. Assuming a $followers array:

$followers = Set::extract($followers, 'Users.User.{n}.{(id|name|profile_image_url|screen_name)}');

The {n} wildcard will allow every numeric key within ['Users']['User'] to survive the extract.
The {(id|name|profile_image_url|screen_name)} keeps only the data that maps to those keys (id, name, profile_image_url and screen_name); therefore my $followers array is reduced to:

Array
(
    [0] => Array
        (
            [id] => XXXXXXXX
            [name] => XXXXXXXX
            [screen_name] => XXXXXXXX
            [profile_image_url] => XXXXXXXX
        )

    [1] => Array
        (
            [id] => XXXXXXXX
            [name] => XXXXXXXX
            [screen_name] => XXXXXXXX
            [profile_image_url] => XXXXXXXX
        )
}

There’s some initial processing overhead, but I’m sure that will be more than offset by iterating through a much lighter array in the various actions.

Integrating Facebook Connect with CakePHP’s Auth component

kettle — Tue, 26 May 2009 01:20:57 +0000

I wanted to be able to leverage all of the advantages of using Cake’s built in Auth component in my latest application; problem was that the application needed to allow for both normal user accounts and Facebook Connect generated user accounts. I struggled for a while to find the most seamless approach, and then it clicked — dynamically set Auth->fields.

First, drop the facebook client libraries in a app/vendors/facebook folder. Then, in your app_controller, import the classes, and set some properties:


Next we’ll overwrite the inherited  __construct method to instantiate a Facebook Object:
	function __construct() {
		parent::__construct();

		// Prevent the 'Undefined index: facebook_config' notice from being thrown.
		$GLOBALS['facebook_config']['debug'] = NULL;

		// Create a Facebook client API object.
		$this->facebook = new Facebook($this->__fbApiKey, $this->__fbSecret);
	}

In our beforeFilter method, we define the default Auth properties, call a private __checkFBStatus method, and pass any user data to the view:
function beforeFilter() {
    // Authentication settings
    $this->Auth->fields = array('username' => 'email', 'password' => 'password');
    $this->Auth->logoutRedirect = '/';

    //check to see if user is signed in with facebook
    $this->__checkFBStatus();

    //send all user info to the view
    $this->set('user', $this->Auth->user());
}

Next we’ll define the private method __checkFBStatus that’s called in our beforeFilter method.  First we check to make sure $this->Auth->User isn’t already set by a normal user account, and we check to see if there’s a logged in facebook user.  The logged in facebook user is available after we use the JavaScript API to log facebook users into the site.
    private function __checkFBStatus(){
        //check to see if a user is not logged in, but a facebook user_id is set
        if(!$this->Auth->User() && $this->facebook->get_loggedin_user()):

Next, see if this user has already logged in, and therefore already has an entry in our User table:
            //see if this facebook id is in the User database; if not, create the user using their fbid hashed as their password
            $user_record =
                $this->User->find('first', array(
                    'conditions' => array('fbid' => $this->facebook->get_loggedin_user()),
                    'fields' => array('User.fbid', 'User.fbpassword', 'User.password'),
                    'contain' => array()
                ));

If no record was found, we create a new record for this user. We are setting 3 variables, fbid, fbpassword, and password.  The fbpassword field will hold a randomly generated 20 character string un-hashed so that we can access the value of the field directly from the database.  We retrieve this value based on the fbid field, hash it, and that’s our password as the Auth Component expects:
            //create new user
            if(empty($user_record)):
                $user_record['fbid'] = $this->facebook->get_loggedin_user();
                $user_record['fbpassword'] = $this->__randomString();
                $user_record['password'] = $this->Auth->password($user_record['fbpassword']);

                $this->User->create();
                $this->User->save($user_record);
            endif;

We need to then update our Auth Component’s fields property to use fbid as the username.
            //change the Auth fields
            $this->Auth->fields = array('username' => 'fbid', 'password' => 'password');

            //log in the user with facebook credentials
            $this->Auth->login($user_record);

        endif;
    }

Here’s the complete app_controller.php:
facebook = new Facebook($this->__fbApiKey, $this->__fbSecret);
	}

    function beforeFilter() {
		// Authentication settings
		$this->Auth->fields = array('username' => 'email', 'password' => 'password');
		$this->Auth->logoutRedirect = '/';

        //check to see if user is signed in with facebook
        $this->__checkFBStatus();

        //send all user info to the view
        $this->set('user', $this->Auth->user());
	}

    private function __checkFBStatus(){
        //check to see if a user is not logged in, but a facebook user_id is set
        if(!$this->Auth->User() && $this->facebook->get_loggedin_user()):

            //see if this facebook id is in the User database; if not, create the user using their fbid hashed as their password
            $user_record =
                $this->User->find('first', array(
                    'conditions' => array('fbid' => $this->facebook->get_loggedin_user()),
                    'fields' => array('User.fbid', 'User.fbpassword', 'User.password'),
                    'contain' => array()
                ));

            //create new user
            if(empty($user_record)):
                $user_record['fbid'] = $this->facebook->get_loggedin_user();
                $user_record['fbpassword'] = $this->__randomString();
                $user_record['password'] = $this->Auth->password($user_record['fbpassword']);

                $this->User->create();
                $this->User->save($user_record);
            endif;

            //change the Auth fields
            $this->Auth->fields = array('username' => 'fbid', 'password' => 'password');

            //log in the user with facebook credentials
            $this->Auth->login($user_record);

        endif;
    }

    private function __randomString($minlength = 20, $maxlength = 20, $useupper = true, $usespecial = false, $usenumbers = true){
        $charset = "abcdefghijklmnopqrstuvwxyz";
        if ($useupper) $charset .= "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
        if ($usenumbers) $charset .= "0123456789";
        if ($usespecial) $charset .= "~@#$%^*()_+-={}|][";
        if ($minlength > $maxlength) $length = mt_rand ($maxlength, $minlength);
        else $length = mt_rand ($minlength, $maxlength);
        $key = '';
        for ($i=0; $i<$length; $i++){
            $key .= $charset[(mt_rand(0,(strlen($charset)-1)))];
        }
        return $key;
    }

}
?>

Next we need to make a few changes to our default.ctp layout; first add the facebook namespace:


Add these 2 javascript snippets (be sure your xd_receiver.htm file is accessible in your webroot):



Last step, place your login and logout buttons in your default view.  Few things to point out here; onlogin="window.location.reload();" will cause the page to reload after a successful Facebook Connect login.  This is how we are able to access $this->facebook->get_loggedin_user() in our app_controller to force the manual Auth login.  If the user is a facebook user (determined with $user['User']['fbid'] > 0), add an onclick event that calls the FB.Connect.logout() function with a redirect URL as the parameter.  This will log the User out of the Auth Component after it logs the user out of Facebook.
 0):
                    echo $html->link('logout', '#', array('onclick' => 'FB.Connect.logout(function() { document.location = \'http://your_server/users/logout/\'; }); return false;'));
                else:
                    echo $html->link('logout', array('controller' => 'users', 'action' => 'logout'));
                endif;
            else:
                echo '';
            endif;
?>

One last thing to consider, for security reasons… In your login method, be sure only users with fbid = 0 can login via the normal auth fields (username / password).  Just an extra precaution considering you have an unhashed password in your database for those facebook users.
Hope that helps someone else out.  Let me know if you run into any problems, I slapped this post up quickly and hopefully didn’t make many mistakes.  And per Matt’s spot-on recommendation (via pseudocoder.com), I’m working on rolling this into a plugin… Will post when it’s complete.

Mobile device detection and development with CakePHP

kettle — Wed, 25 Mar 2009 22:26:28 +0000

I was recently working on a project that required a mobile version of the site (built with cakePHP). The site was simple enough that the majority of the actions that rendered views were contained within one controller. Using cake’s built in Request Handling makes it easy to detect mobile devices and select an appropriate view and layout file to render the action.

In your app_controller.php, turn off autoRender if the isMobile method returns true:

// file: /app/app_controller.php
class AppController extends Controller {
    var $components = array('RequestHandler');

    function beforeFilter(){
        if ($this->RequestHandler->isMobile()) {
            $this->autoRender = false;
        }
    }

    function afterFilter(){
        if ($this->RequestHandler->isMobile()) {
            $this->render($this->action, 'mobile', 'mobile_'.$this->action);
        }
    }
}

The afterFilter function runs the same conditional statement as beforeFilter, but tells cake to render the currently running action using the ‘mobile’ layout, using the ‘mobile_action’ view. So when you create your normal view files, also create a mobile version with the same file name, preceeded with ‘mobile_’. This approach assumes you want to run the same controller logic for both versions of the site (mobile and standard). You could always use routes to create a more elegant system for handling requests from mobile devices, but for this particular project, the logic didn’t change much beyond defining the default limit with pagination.

And don’t forget, if you’re defining afterFilter and beforeFilter in your app_controller, be sure your controllers inherit app_controller’s filter functions.

// file: /app/controllers/posts_controller.php
class PostsController extends AppController {
    var $name = 'Posts';

    function beforeFilter(){
        parent::beforeFilter();
        //rest of your beforeFilter function here...
    }
}