Benchmarking the Outsourcers

By Jesse Freund

If managed correctly, outsourcing can provide value and improve productivity. This trend was made evident in a recent Gartner Group report, Gartner on Outsourcing, Q404. In it, the researcher found that the primary motivation for outsourced development has been shifting from cost-only to cost-and-quality, as well as to cost-and-business impact. As outsourcing clients grow more concerned with quality, expertise, and cultural compatibility, Gartner expects that the extreme focus on cost savings will moderate. The report concludes that the market for outsourced application development will grow from $34.9 billion in 2003 to $47.5 billion by 2008.

Despite the growth and maturation of the outsourced market, however, nearly 30 percent of the respondents to a recent Information Week survey reported some level of dissatisfaction with an outsourced vendor. To mitigate the risk of an outsourcing relationship going awry, there are a number of steps smart CIOs need to take to solidify an outsourced relationship already in progress.
 
Set benchmarks early

"The biggest pitfall I see when it comes to outsourcing is giving up control too early in the development process," said Sameer Patel, principal and lead strategist at SpanStrategies, Inc., a consultancy that helps several Fortune 500 companies manage outsourced engagements. "Customers quickly become too comfortable with their vendors and believe that the vendor will take care of everything from soup to nuts. If you don't want any surprises when you flip the switch, you can't come up with fuzzy ideas and then hand them over the fence."

To avoid unwanted surprises, Patel helps his clients develop a detailed sense of the final product before the vendor begins engineering. Patel's engineering roadmap, or benchmarks, typically includes three components:

• Clearly articulated business requirements that discuss the business need behind the development.
  
  
• A working prototype that demonstrates the usability design and removes any guesswork from the intended user experience.
  
  
• A detailed functional specification that explains all user interactions.

These three items should be frozen and delivered to the vendor before coding begins. The idea behind a structured roadmap is to ensure that a product at the end of the development cycle addresses the business needs that necessitated the development to begin with.
  
How much benchmarking is enough?

Asked for advice on how benchmarking can help ensure a fruitful vendor-client relationship, Gartner analyst Robert Brown replied, "Do it early, and do it often."

The reasons for benchmarking are many, but essentially the process is essential for evaluating the client-vendor relationship in order to pinpoint problems and maximize potential. Benchmarking can help a client identify strengths and weaknesses in a relationship, define best practices, and fix faulty processes. It can also help a company decide when it's time to adjust a relationship or even end it. And, finally, in the event of a relationship gone awry, benchmarking can help a client select a more suitable vendor the second time around. With benchmarks clearly articulated and defined in advance, making such decisions becomes easier.

To benchmark effectively, companies should begin the process before a deal is signed. Businesses need to measure the current output and compare that to the cost of development. For example, suppose a company is considering developing an extranet for its sales force. The company must examine the current performance of its sales team, quantify the improvement in leads it is seeking, and evaluate how much that will cost per lead. Oftentimes, enterprises lack the expertise to perform such an audit in-house, in which case a third-party firm, such as Gartner, can be engaged to perform the measurements.

Benchmark the right stuff

In any deal, relevant metrics must become an integral part of the Service Level Agreement (SLA). Most companies know that availability and performance need to be benchmarked. And, indeed, 99.9 percent up-time and a fast load time are important. But when it comes to benchmarking, companies need to think beyond these bare minimums. It's important to turn business requirements into technical metrics. For example, if the new sales extranet is supposed to increase lead generation by four percent, that metric should be specified in the SLA. Similarly, things like customer satisfaction and quality assurance responsibilities need to be examined, turned into relevant metrics, and specified in any signed deal.

Continue to benchmark often

Even as an outsourced relationship is under way, companies need to continue to measure performance to see how the relationship is playing out. Businesses should schedule monthly or quarterly meetings to discuss milestones and performance. If a vendor isn't hitting its metrics, regular meetings can help identify the shortcomings and allow time to come up with strategies to address the problems. It's a good idea to have a dedicated team that conducts the measurements and meets with the vendor, but it's an even better idea to have a higher-level management team that reviews the work of the dedicated team.

There are at least two positive outcomes of benchmarking. First, most vendors realize the importance of benchmarking and are happy to participate. (If a vendor isn't willing to be held to benchmarked standards, it's time to find a new vendor.) Second, in the event that a vendor isn't living up to its end of the bargain, benchmarking should not only reveal any deficiencies but also help a company select a more appropriate vendor the second time around.

Jesse Freund is a contributing writer at Business 2.0 and a frequent contributor to Wired.