Which is good, as I'm doing a talk at SkillsMatter in December on that very subject.

Since I won an iPhone in a raffle last month I now have a surplus ipaq that used to be my portable brain, randomly I came across this site on running Linux on ipaqs. So this maybe the way to go.

However this guy also made some really cool bots running windows C++ code. I think certainly with an ipaq as the brains plus computer junk from the loft I probably have the kit for some interesting junkbots.

Tesco is starting the rollout of it's new backend Grocery application, code named Martini (anytime, anyplace, anywhere).
They have opened up their new APIs from their grocery service to a few select developers. In fact very select, you had to attend to launch/briefing in person to register for the scheme.

A lot of time had obviously gone into preparing the event. With almost the whole conference room covered in brainstorm sheets and post it notes. These covered 5 main areas for potential developers to concentrate on or consider for their application. This was based on market and socioethnic research (hey that may not be the correct word but I didn't write it down and I'm now on the train).

After an initial introduction, we were all invited to chow down on pizza, lager and wine (actually nice for change to get red and white wine available) and browse the brainstormings, only slightly hindered by Microsoft's insistence that no one take food or drink into their meeting rooms.

The APIs look reasonably well thought out and the new Martini release adds some great new functionality, such as nutrition information and cheaper alternatives to products in your shopping basket.

There were some concerns voiced by some developers present at the briefing, quite rightly, over their security implementation, which requires the API consumer to forward the users login credentials, rather than implementing OAuth or another signature/token based authentication mechanism.
It looks like this will probably be addressed in a future release, once Martini is live and running Tesco's existing shopping site.
Customer basket checkout has yet to) be finalised, due to financial and legal compliance.

Everyone who attended the event and wishes to participate in the affliate scheme will get signed up to TradeDoubler and will earn £5 for each new customer they manage to entice into signing up to Tesco's online shopping service.
The tricky part of this is then the ongoing revenue.
For each basket purchased over the value of £50 the application developer gets a potential £0.10. Potential, as this is the proportional to the number of products added to basket by that particular application versus the total number of products in the basket.
Note also products in basket not items. 10 one penny chews would earn you the same as someone adding an iPhone or flatscreen tv. This is deliberate, to prevent skewing of apps towards higher value goods.

In the last post we looked at the internals of CodeSniffer and how the processing of files is done. This time we will look at how an individual Sniff is constructed.

Sniff Basics

• Each Sniff is a PHP class.
• Each class consist of a minimum of two methods.
  • A register method, that returns an array of tokens that the Sniff is interested in.
  • And a process method that is executed each time that token is encountered.
• Each Sniff class lives inside a Standard.
  • Each Sniff lives inside a subfolder "Sniffs" under the standard
  • Each Sniff lives inside a subfolder of "Sniffs" containing themed Sniffs
• Each Sniff class must adhere to the naming convention.

Starting a Sniff

Let's take a look at an example Sniff. For instance a Sniff designed to check that our PHP code has no extraneous whitespace outside of our PHP code, that could cause a PHP error should we subsequently send headers to the browser.

First of all we create a class and decide on a name for it.

As CodeSniffer uses an autoloader that loads the class' file and path based on the class name we have to stick to a specific naming convention.

class KingKludge_Sniffs_PHP_NoExtraneousWhiteSpaceSniff
implements PHP_CodeSniffer_Sniff {}

equates to a path of:

{CodeSniffer}/Standards/KingKludge/Sniffs/PHP/NoExtraneousWhiteSpaceSniff.php

where {CodeSniffer} is your CodeSniffer install path.

Secondly, inside our new class we must define our register method:

public function register()
{
	return array(
		T_OPEN_TAG,
		T_CLOSE_TAG,
		);
} 

As we want to check for whitespace before or after our PHP declarations, we state that this sniff is interested in PHP open and close tags.

Thirdly, we must define our process method:

public function process(PHP_CodeSniffer_File $phpcsFile, $stackPtr)
{

}

As you can see, our process method is passed some parameters.

These are $phpcsFile which is PHP_CodeSniffer_File object, containing the file being processed and $stackPtr an integer value pointing to CodeSniffer's current place in the file.

View the empty standard file on GitHub

Now on to the actually code in our Sniff.

As our proposed Sniff is going to look for whitespace being output before our page is ready to be rendered, we are interested in whitespace that occurs before the opening PHP tag or after the closing PHP tag.

We are only interested in whitespace, any HTML or other markup is okay.

So we can see if our pointer is at the beginning of the file, if so, drop out of the process.

$tokens = $phpcsFile->getTokens();
if (T_OPEN_TAG === $tokens[$stackPtr]['code'])
{
	if (0 === $stackPtr)
	{
		return;
	}
}

The getTokens() command returns all of the tokens for the current file in an associative array. (in case you didn't realise).

The rest should be self explanatory.

We can also do a similar check for a PHP close tag

if (T_CLOSE_TAG === $tokens[$stackPtr]['code'])
{
	$content = $tokens[$stackPtr]['content'];
	if (
	    false === isset($tokens[($stackPtr + 1)]) &&
	    trim($content) === $content
	   )
	{
		return;
	}
}

View the basic standard file on GitHub

This article will try to cover in-depth how CodeSniffer actually works to give insight into the next proposed article, writing a sniff from scratch.

The PHP Tokenizer

CodeSniffer works by extending the PHP tokenizer function.

Given the following section of code:

<?php

function DoSomething(array $foo) {
    print_r($foo);
}

?>   

and running it through PHP's native tokenizer we get the following output

PHP Tokenized version of foo.php

PHP's tokenizer only identifies a limited subset of PHP syntax as listed here.

All other potential tokens get either identifies as a string, signified by the sub array[0] integer value of 307 or the constant T_STRING, else it simply returns the string value of the token i.e. those seen array values 17,18 and 20 in the sample output above.

To map the above integer values to PHP's string constant name you can use the PHP function token_name()

For example:

$ php -r "print(token_name(369));"
T_CLOSE_TAG

As once the PHP tokenizer has run, we have a lot of code still encapsulated as T_STRING or with no tokenizing done, CodeSniffer takes these simple tokens and expands them further.

CodeSniffer introduces new constants such as T_TRUE, T_FALSE, T_NULL, T_PARENT, T_OPEN_CURLY_BRACKET and so on.

This gives CodeSniffer considerable scope to be able to handle much finer detail of the PHP syntax.

The PHP CodeSniffer Tokenizer

When CodeSniffer first loads, the standard in use is determined from the command line or from the stored config. The standard is then loaded and all of that standard's sniffs are loaded.

Each of the sniffs gets called via the register() method and a hash of all the tokens and classes is created.

Then CodeSniffer starts looking for the files to check, if a directory is specified, CodeSniffer iterates through the directories until a file with the correct extension is found, then each file is processed in turn.

If a list of files or a single file is specified, then the above step is skipped and CodeSniffer starts parsing the file(s) as defined in the parameters.

Once CodeSniffer has tokenized the file under analysis into one (rather large) multidimensional array of language syntax tokens, the rest is quite simple.

CodeSniffer breaks each file under examination down and does a series of context checks before processing the tokens and calling all the registered sniffs.

These checks are:

1. Bracket Map: checking braces
2. Scope Map: checking for class, function and conditional statement scopes
3. Level Map: checking for class, function and conditional statement levels

If we look at the CodeSniffer Tokenized version of foo.php we can see the levels of our sample script above.

Each Sniff in the standard has registered which tokens they are interested in being invoked to handle during the initialisation phase.

CodeSniffer then runs through each token in the file from beginning to end and calls all of the sniff process() method for sniffs that registered and interest in that token.

Finally all of the errors and warnings generated by those sniffs are organised into the desired report type and displayed.

So now we have an insight into how CodeSniffer works, in the next and final post in this series on CodeSniffer, we'll look and writing a new Sniff.

$ phpcs --report=source --standard=Zend ./_rr

PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
--------------------------------------------------------------------------------
STANDARD    CATEGORY            SNIFF                                      COUNT
--------------------------------------------------------------------------------
Zend        Files               Line length                                92
Generic     White space         Disallow tab indent                        65
Zend        Naming conventions  Valid variable name                        59
PEAR        White space         Scope closing brace                        33
PEAR        Functions           Function call signature                    26
PEAR        Control structures  Control signature                          12
PEAR        Files               Line endings                               11
PEAR        Functions           Function call argument spacing             5
Generic     PHP                 Disallow short open tag                    1
Zend        Files               Closing tag                                1
--------------------------------------------------------------------------------
A TOTAL OF 305 SNIFF VIOLATION(S) WERE FOUND IN 10 SOURCE(S)
--------------------------------------------------------------------------------

Each CodeSniffer standard is comprised of rules or sniffs. A standard can contain a sniff from other standards.

This allows us to quickly and easily create our own custom standard by leveraging those rules that are already contained in existing standards.

To find out what we have available to use, we can looking for all the files that end in Sniff.php within the Standards folder. We can see we have 167 sniff available to use already (with version 1.2.0a1 anyway).

Full list of available Sniffs

cd {your pear path}/PHP/CodeSniffer/Standards
$ find ./ -name "*Sniff.php" -and -not -name "Abstract*"

Clicking here to see the full list of available Sniffs.

Some of these Sniffs are self explanatory some are less so, and would need a cursory glance at the code to see what they are checking  for.

Luckily most of the classes are well documented (as the PHPCS standard dictates!) so looking at the class phpdoc comment usually tells what the sniff is looking for.

Something to note is that you will find some of the sniffs are mutually exclusive:

e.g.

./Generic/Sniffs/Formatting/NoSpaceAfterCastSniff.php
./Generic/Sniffs/Formatting/SpaceAfterCastSniff.php
./Generic/Sniffs/Functions/OpeningFunctionBraceBsdAllmanSniff.php
./Generic/Sniffs/Functions/OpeningFunctionBraceKernighanRitchieSniff.php

Writing the standard

So to start with we need to come up with a name for the standard.

So for this example I'm going to use KingKludge.

Now I prefer to develop in my home dir, but to get this sniff recognised you will need to have the standard available in {your pear path}/PHP/CodeSniffer/Standards/.

So I symlink my standard into the pear path, but you could work directly in the pear folder.

$ mkdir KingKludge
$ ln -s KingKludge {your pear path}/PHP/CodeSniffer/Standards/KingKludge

Then we need to create the standard file. Which follows the naming convention {standard}CodingStandard.php so we end up with the following file.

KingKludgeCodingStandard.php

The underscores in the class name are important as the autoload function splits the path at underscores, so if you miss-type one of the path parts you will get a fatal error when the class attempts to load.

Assuming that you did have your class in the correct folder you can do run.

$ phpcs -i

And you should see your new standard is listed as available for use.

As a timesaving tip, to stop us from having to type the standard name every time you can run CodeSniffer run the following command.

$ phpcs --config-set default_standard KingKludge

Now unless we override with the --standard switch CodeSniffer will default to using the KingKludge standard.

Adding some rules

So now we have a new standard but it doesn't do anything yet, as it has no rules to apply.

We have two methods in the class to define the rules or sniffs getIncludedSniffs() and getExcludedSniffs(). Not unsurprisingly these methods should return an array of the sniffs to include or exclude.

If we decided we wanted to make life easy for ourselves and base our standard on someone else's standard, we can quite simply include all of their standard and then swap out the sniffs we don't want for others we do.

For example, taking the Squiz coding standard.

public function getIncludedSniffs()
{
  return array(
    'Squiz',
  );
}

Now say we want to swap the BSD style bracing rule for K&R style braces, and we don't want the CodeAnalyzer rule to run either.

So we add the K&R braces Sniff to our include and add the CodeAnalyzer and BSD braces sniffs to the exclude.

public function getIncludedSniffs()
{
  return array(
    'Squiz',
    'Generic/Sniffs/Functions/OpeningFunctionBraceKernighanRitchieSniff.php',
  );
}

public function getExcludedSniffs()
{
  return array(
    'Generic/Sniffs/Functions/OpeningFunctionBraceBsdAllmanSniff.php',
    'Zend/Sniffs/Debug/CodeAnalyzerSniff.php',
  );
}

That seems easy enough!

So now you can see how easily you can create your own standards by picking as few or as many sniffs as you want from existing standards.

Next time, understanding the internals of CodeSniffer and how it works.

If we take an example block of code such as the following.

 
 
<?php

function DoSomething(array $foo) {
    print_r($foo);
}

?>
 

And run it through CodeSniffer with the Zend coding standard we get:

$ phpcs --standard=Zend foo.php

FILE: /home/bob/sample/foo.php
--------------------------------------------------------------------------------
FOUND 2 ERROR(S) AND 1 WARNING(S) AFFECTING 2 LINE(S)
--------------------------------------------------------------------------------
 2 | ERROR   | End of line character is invalid; expected "\n" but found "\r\n"
 4 | ERROR   | Opening brace should be on a new line
 4 | WARNING | Consider putting global function "DoSomething" in a static class
--------------------------------------------------------------------------------

If we then run the same code through the PHPCS standard we get a more in-depth set of violations:

$ phpcs --standard=PHPCS foo.php

FILE: /home/bob/sample/foo.php
--------------------------------------------------------------------------------
FOUND 16 ERROR(S) AND 2 WARNING(S) AFFECTING 6 LINE(S)
--------------------------------------------------------------------------------
 2 | ERROR   | End of line character is invalid; expected "\n" but found "\r\n"
 2 | ERROR   | End of line character is invalid; expected "\n" but found "\r\n"
 2 | ERROR   | Additional whitespace found at start of file
 3 | ERROR   | Missing file doc comment
 4 | WARNING | Consider putting global function "DoSomething" in a static class
 4 | ERROR   | Missing function doc comment
 4 | ERROR   | Function name "DoSomething" is invalid; consider "doSomething"
   |         | instead
 4 | ERROR   | Expected "function abc(...)\n"; found "function abc(...) "
 4 | ERROR   | Function name "DoSomething" is not in camel caps format
 4 | ERROR   | Expected 2 blank lines before function; 1 found
 4 | ERROR   | Opening brace should be on a new line
 4 | ERROR   | Opening brace should be on a new line
 4 | ERROR   | Opening brace should be on a new line
 5 | WARNING | The use of function print_r() is discouraged
 6 | ERROR   | Expected //end DoSomething()
 6 | ERROR   | Expected 1 blank line before closing function brace; 0 found
 6 | ERROR   | Expected 2 blank lines after function; 1 found
 8 | ERROR   | Additional whitespace found at end of file
--------------------------------------------------------------------------------

Hopefully you can see that the PHPCS standard is significantly more strict that the previous.

Also you should see that apart from more style based checks, this standard also checks for whitespace at the beginning and end of the file. Whitespace being echoed can be very important for some PHP frameworks. When lots of files are included as part of the execution, especially if they are dynamically loaded, the whitespace can cause a fatal error if header information is also being sent to the browser.

The other issues contained in this report are based around the project code layout and documentation requirements.

Other reports

With the release of CodeSniffer 1.2.x there are some new reporting types available.

Rather than the full in-depth report, two reports that I find quite useful for gauging a project's status are the summary and source reports.

Summary reports the number of errors and warnings found on a file by file basis over a directory tree.

$ phpcs --report=summary --standard=Zend ./_rr

PHP CODE SNIFFER REPORT SUMMARY
--------------------------------------------------------------------------------
FILE                                                            ERRORS  WARNINGS
--------------------------------------------------------------------------------
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr1.php  23      3
...yocal/apps/frontend/modules/_rr/templates/_partial_rr10.php  4       1
...yocal/apps/frontend/modules/_rr/templates/_partial_rr11.php  7       4
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr2.php  8       2
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr3.php  18      2
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr4.php  43      3
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr5.php  22      4
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr6.php  39      3
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr7.php  24      1
.../yocal/apps/frontend/modules/_rr/templates/_partial_rr8.php  17      2
...yocal/apps/frontend/modules/_rr/templates/_thumbsUpDown.php  71      4
--------------------------------------------------------------------------------
A TOTAL OF 276 ERROR(S) AND 29 WARNING(S) WERE FOUND IN 11 FILE(S)
--------------------------------------------------------------------------------

Source shows the  type of errors and warnings found in a project:

$ phpcs --report=source --standard=Zend ./_rr

PHP CODE SNIFFER VIOLATION SOURCE SUMMARY
--------------------------------------------------------------------------------
STANDARD    CATEGORY            SNIFF                                      COUNT
--------------------------------------------------------------------------------
Zend        Files               Line length                                92
Generic     White space         Disallow tab indent                        65
Zend        Naming conventions  Valid variable name                        59
PEAR        White space         Scope closing brace                        33
PEAR        Functions           Function call signature                    26
PEAR        Control structures  Control signature                          12
PEAR        Files               Line endings                               11
PEAR        Functions           Function call argument spacing             5
Generic     PHP                 Disallow short open tag                    1
Zend        Files               Closing tag                                1
--------------------------------------------------------------------------------
A TOTAL OF 305 SNIFF VIOLATION(S) WERE FOUND IN 10 SOURCE(S)
--------------------------------------------------------------------------------

As you can see from this report, although we are using the Zend standard, it consists of individual rules from multiple standards.

In the next part of this series I will cover how to write your own standard, by selecting rules from existing standards.

It ended up way too long,  so I have split it up into sections and will post as a series.

This is a rough transcript of a lightning talk I gave during a team meeting at Yahoo! completely off the cuff, but I am in the process of writing this up as a Yahoo! Tech Talk, and also, as this is not internally sensitive, like most of the stuff I've been working on for the last two years, I'll see if it can be release on Yahoo Developers Network.

What is CodeSniffer?

CodeSniffer is almost a SCA tool. It allows a set of rules or a Standard to be applied to source code.

These rules can be used to detect common programming errors or Anti Patterns, it can also be used to define a set of Coding Standards  for your project.

It is down to your project to come up with some Coding Standards or apply a pre-existing set of standards for your team to adhere to.

CodeSniffer comes with a set of coding standards already defined that are regularly used by other teams.

These are:

• MySource
• PEAR
• PHPCS
• Squiz
• Zend

Analysing your code using one or more of these standards will highlight how the code you have provided stacks up against the project's coding standard.

How to install CodeSniffer

$ sudo pear install PHP_CodeSniffer

This will download and install the CodeSniffer utility from the PEAR repository.

How to run CodeSniffer

Simply run:

$ phpcs --standard=<standard> <path to file or directory>

You will be shown a list any errors or warnings in your code.

CodeSniffer will also exit with a non-zero status code. This can be useful if you are using a Makefile to test your code or some build tool like Phing, or if your CodeSniffer rules are being run as part of a Continuous Integration system.

In the next post I will run through some of the standards and reports.

This is a regular occurance on may different sites and generally seem to aimed towards financial services (a.k.a. banks) and high profile online shopping (ebay, amazon and the like).

What makes this more of a big deal is that generally people have a hard time remembering passwords, so tend to use the same password for everything,  bank(s), ebay,  paypal, twitter, flickr, yahoo!, google, facebook, etc..  So if you get the username and password for one user, you may well be able to compromise that user all over the web.

However one thing that you can do to protect your web presence is NOT USE THE SAME PASSWORD EVERYWHERE!

Now I use a firefox plugin called PasswordMaker.

This creates a hash of a password of your choice with the domain name of the site you are currently browsing and will add a content menu link to your browser to allow you to easilt populate web forms when you need your password.

It has lots of options, but here's a quick screen grab of the main page:

PasswordMaker main panel, showing generated password for this site. No not my real one, you think I'm dumb!

As you can see, using the master password, i.e. your one password you can remember, it will generate a unique password for each website you visit.

You can also cache you master password for one browser session (store master password: in memory) or permanently (store master password: on disk & in memory).

You can choose the password length, hashing algorithm, obfuscator (i.e. leet), character set to use (see below).

PasswordMaker Settings

All of these can be altered for specific sites, e.g. use longer password lengths for banking sites etc..

If you're not a fireFox user, there is a downloadable version for your desktop, or a JavaScript version you can save, or an online version (which also uses JavaScript they don't post your details to a service! That would be nuts!).

PasswordMaker is a SourceForge project, so if you want to look under the hood, lend a hand or even get ideas for other neat ways to do something similar, you can.

I suspect we will see more of this social sites targeted for phishing as people are getting wise to the paypal, ebay, bank phishing and more browsers or browser toolbars are detecting these and blocking them. Leaving social sites that are perceived as low threat, for now, as a possible exploit vector.

So use different passwords for different sites, or you may be liable to one phish compromising your whole online life. There are tools to help!