Key Performance Indicators

What is the scenario in India when it comes to SLA management?

We believe that the adoption of formal SLAs is fairly limited in India. SLA formulation and management requires IT and multiple business stakeholders to accept a single version of the truth, in terms of the data that drives the metrics. This is typically a fairly resource-intensive task that involves aggregating data from multiple application repositories, databases, infrastructure management systems, configuration management databases and service desks. The cost is at least in the order of tens of thousands of dollars, and in many cases above the U.S.$100,000 mark.

A company would take up such a resource-intensive project only if sufficient scale exists and/or if the awareness of the need for IT maturity is high – these conditions are rare in India. Even globally, formalized internal SLAs are more important to the very large enterprise than any other kind of company (this does not include service providers of all kinds, because SLAs are the core of their business).
When defining an SLA with a service provider, what are the key parameters to be specified?

That really depends on the provided service. Any infrastructure management (including outsourced help desk) sort of service would have SLAs that cover incident response time (including tasks such as adds, moves and changes), mean time to repair, availability (such as network and email availability) and how these variables trend over time. A lot of work usually gets done before parties can agree about defining the right metric and context for each (such as, which event would define incident closure?), segmenting types of events (based on severity), periodicity of monitoring, data collection and aggregation procedure, rules defining penalties and the escalation procedure.

When the service involves application hosting, SLAs would be related to uptime (availability), which is qualified by factors such as the number of concurrent users to be expected. Performance is a little more difficult to define. Ideally, performance should be measured in terms of end-user experience, but sometimes measuring end-user experience involves hard-to-scale tasks such as installing agents on end users' desktops, and with the growth of mobile users, it's hard to control the endpoint and the network, complicating matters.
What are the aspects to keep in mind when it comes to managing an SLA?

SLAs are not substitutes for a good partner relationship and governance procedures. Therefore, a broader management scope would still be necessary. Inordinate effort to make the SLA absolutely watertight isn't necessary, particularly when the internal IT organization involved. Select the SLA that best represents the end user's actual experience, while being realistic about what the expectations should be. So, application availability makes much more sense than server uptime does.

It needs to be appreciated that SLAs can be met only under certain given conditions – for example, if the average user's Web experience is to remain above the threshold, strong URL filtering is required to ensure that Web usage is in accordance with policy, and requests for exceptions should be carefully managed. At a high level, there isn't much that is arguable about the right SLAs. However, collecting and aggregating metrics is usually quite tough. Data typically resides in multiple databases, infrastructure management systems and possibly multiple service desk solutions. Building connectors to these data sources, aggregating them and developing dashboards for reporting are all resource-intensive tasks.

Tools don't do much here – the professional services' costs typically equal the average SLA management tool's licensing costs (a 1-to-1 ratio). Hiring an expert who is familiar with the metrics that work in terms of securing buy-in (and truly representing the business' interests) and hiring the IT talent to build the data connectors is much more important than any tool-related considerations.
How can IT teams leverage the various features offered by SLA management tools?

SLA management tool vendors offer value in multiple ways. First of all, they know which metrics to select. They also know where to look for data necessary to build metrics that IT and business would monitor as part of the SLA.

Vendors have the necessary tool knowledge to build connectors to the data. They also provide mechanisms such as rules engines that ease the process of aggregating the data to build metrics to be monitored. Dashboards are usually provided to aggregate and present the metrics in an automated way, which provide automated alerting services, analytics, etc., and in effect, create a single system of records that everybody can agree on. So, it's less about tool features and more about competence that the SLA management tool provider brings to the table.
Somak Roy, managing analyst at Butler Group, Datamonitor India, gives his views on service-level agreement (SLA) management.

You tried a little junior college before deciding to skip higher education and go to work. Do you have any formal training in computer science?

I did take a course in COBOL, which was extremely useful, mainly because I saw that not everybody could do something that came pretty naturally to me. I discovered that everybody is good at something. You just have to figure out what it is.
You started out as a computer operator, at 18, at Computer Sciences Corp.

I got in trouble real bad. Because I could sit there and just operate the thing, I started logging on and trying to snoop around. But instead of being fired, I got promoted and I got a wonderful opportunity to work in another division, which was working on something they called DNS but was actually the very early stages of client server technology. I was developing database applications. I was exposed to a variety of customers.
Like who?

St. Jude's Children's Hospital came to us and said we'd like to use your computer and could you help us build an application that would help us keep track of all of our donors. I was behind the scenes developing this application according to spec. When it came time to turn it over, I was brought in and went to train people on it. Bless their hearts, there are these two little old ladies who were afraid of the computer.

I always hark back to that. Here I was behind the scenes having a blast designing this database, thinking about how to make it more efficient and all this other stuff, but I realized none of that made any sense to these ladies and they didn't care. In my career I have seen the habitual problem that IT has of not understanding the business value, and very early on in my career I had an opportunity to see that problem.
What was your worst job?

I was working at a major financial institution with a 700-person IT shop. You got lost. It was tough to accomplish anything. It was around that time I realized I really am a doer. I can get bored. I remember the day when I came in and cracked open a newspaper like everybody else did and ended up reading it cover to cover I said, 'I can't do this. This isn't me.'
How did you get into the entertainment business?

I had left CSC and was working at the financial institution and various other things and came back to Computer Sciences. Then one day I got approached by a headhunter about a job at MGM United Artists. I started off as a manager over the financial systems and became a director of applications and development. When I took that, it rejuvenated me about what I was doing.
Is there any entertainment experience in your background?

In junior college, I worked in theater arts behind the scenes. I did publicity, lights, sound, stage managing. I am a musician. I have a studio in my own house. I have a Christian rock band. We play in boy's prisons. We even played a Christian biker festival.
Getting back to your career, what's the best career advice you've gotten?

Don't argue with a fool because somebody walking into the middle of the conversation won't be able to tell you apart. IT is a strange business. People, for example, don't call you up and thank you when however many thousands of users are on your network are able to log in today successfully. They only call you when they can't.
So the enabler rarely gets to bask in the success.

We become the go-to source, and that has a good and a bad side to it. They're always running to us and complaining, but I started to realize that they're running to us because we are the geeks, or whatever you want to call us, within the organization that people are looking to and trust will be able to solve their problems. Inherent in that is the thank you.
Tell me about a good CIO decision you've made recently.

When our data warehouse went live, the first people that were going to receive the reports were our store personnel, not the executives. The week that store system went live, our store managers ran with that ball. We have graphs that show all the key performance indicators in each store. And the store managers are excited. If that system has a minor hiccup we hear about it immediately. They're out there tracking the horse race [sales] every day.
Can one store see what the other stores do?

I've worked in other environments where they are so protective of data. But in our case, we let any store see what the other store's performance is, down to department, down to a SKU, down to a 15-minute increment.
How do you do data management?

We use the Microsoft SQL Server for our data warehouse. I brought together a user team to go out and evaluate business intelligence technologies and ultimately pick. We came down to a bake-off between Hyperion Essbase [TK] and Microsoft's SQL Server. Behind the scenes I had been doing my homework and realized the way SQL Server was priced and the tools that came with it blew others away. One day the team asked me what I was voting for and I refused to answer them. They laughed, and said, 'We knew you would do that.' They made the choice.
So music is an avocation. What's your favorite guilty pleasure?

Golf.
Your handicap?

My entire game. But I play anyway.
What technology do you wish you lived without?

I wish I did live without mobile e-mail.
Are you worried about BlackBerry service being shut down?

I chose not to go with BlackBerry as a standard for our organization. We're using the various Windows Mobile-based or Palm devices. We ourselves at Virgin certainly have been approached about patent infringement, which we've tended to walk away from it pretty unscathed. But I understand the right of the guy who truly created the technology to come back and ask for his just due. It would seem foolish to me that would cause the service to come to a halt.
Robert Fort was in kindergarten when his mother, an applications developer, started taking him to work to help sort punch cards. At 8, he dressed up as a computer for Halloween. After graduating high school a year early, he skipped college and took a job at Computer Sciences Corp. Self-taught and self-assured, the 46-year-old Californian got his big break when he went to work at Metro-Goldwyn-Mayer Inc. Now, as director of IT at Los-Angeles based Virgin Entertainment Group Inc., the North American subsidiary for the U.K. conglomerate, Fort keeps IT rocking at the $200 million company, recently bringing the sales data for every store online to managers throughout the 17-store Megastore chain. We spoke by phone about his vocation and avocations.

How do you pick a channel partner?

We have very specific guidelines that we follow when selecting channel partners. Those guidelines typically revolve around how quickly they can deliver, the breadth of their offering and pricing.
What do you buy and from whom?

Because of the big differential between what we spend on storage versus the rest of our IT needs, we typically look to vendors that handle the largest breadth of gear as possible. For EMC gear, for example, we work very closely with Dell. EMC has one of the largest networks of resellers and VAR [value added reseller] partners out there but we have found that we still get better service from a firm like Dell because of the fact that we buy so much other nonstorage-related equipment, servers, etc.

For smaller purchases or purchases we need in a faster delivery window, we work with a variety of local and national VARs and in many cases buy direct. In the case of software, we will, whenever possible, buy direct if the firm offers direct download. Even when a VAR can offer better pricing, the convenience of a direct download or automatic license purchase is far more important.
In your opinion, what are the advantages of working with a channel partner?

As an organization, we find that we need to be able to have established relationships with a variety of vendors and resellers to meet all of our needs. This is especially true when we are on a tight delivery timetable. A local reseller may charge a premium versus a larger national reseller or the vendor themselves, but they are able to add value and justify that price with a quicker turnaround.

The second key issue is that when we work with the larger resellers, we are able to get a lot better service because of our larger purchase volume. For example, we recently purchased an entry-level EMC SAN for a specific project, and had the scope of our project change overnight. We needed a SAN that could scale much larger and had to go with a higher end model that would scale further. EMC has a policy that once a PO [purchase order] is signed, they will not take a unit back unless it is due to a technical problem. Because of our relationship with Dell, they were more than willing to work with us, and did so in as painless a way as possible. If I was working direct or with a VAR that only handled storage or that I did not have the same dollar volume with, I am not confident we would have had the same experience.

Third, in theory, channel partners can introduce us to new products that we were otherwise unaware of, but in practice that is pretty rare in the current business climate.
What are the downsides to working with resellers?

This depends largely on the specific reseller and the way in which the manufacturer themselves has setup the reseller program. We have seen some nightmare scenarios when resellers are responsible for first tier support or sales engineering and the resellers are not equipped with the necessary technical resources to meet our needs. This is especially true when the reseller does not have the right number of specific subject matter experts and tries to utilize more generalists.

We also have a hard time with resellers that do not stock product. In the current climate, we typically make final purchasing decisions a week or two before we need the product installed in our data center. In some cases, due to project scope changes or emergency capacity upgrades, we need to be able to have a product the next day. Most resellers today, and for that matter, many vendors themselves, are not prepared to meet that kind of delivery timetable. For organizations with static needs or even static rates of change, that may be fine, but for a rapidly growing Internet-centric business it causes a lot of problems. To resolve this issue, we typically have three or four key resellers that can source any one product we use, so with a little legwork we are able to meet our timetable.
Are channel partners really independent?

Absolutely not. In my experience, channel partners are sales organizations and we treat them as such. We have had countless interactions with technology resellers that pitch either the product they have the highest margin on, or a product they need to move for internal purposes. We address this by doing as much homework as possible ahead of time and then utilize their subject-matter experts for specific implementation or integration questions.
Do you think you can get a better price working with a channel partner versus dealing with vendors directly?

My experience with pricing has been that it is directly related to the size of the manufacturer. We find that vendors with a large, diverse, mature product line, like EMC or Sun, offer their best pricing through the channel. Newer players or vendors rolling out a new product and looking for early adopters will inevitably offer better pricing and terms directly, than through their partners. With the competitive VAR landscape, there are fewer firms willing to distribute products on a loss-leader basis, but that still remains a vital strategy for vendors attempting to push a new product into the market place. Like any savvy buyer, we try to exert our pricing pressure and take advantage of such situations.
Buying storage gear from the channel needn't be as dicey as you might imagine as long as you know what to look out for. Aaron D. Sawchuk, co-founder and chief technology officer of managed services provider ColoSpace Inc., in Rockland, Mass., talked with SearchStorage.com about his experiences buying from the channel and some of the advantages this has brought him in dealing with key suppliers.

walking the walk with customers is key to provisioning IT business service ideas successfully into effective moneymakers...

because of its complexity and the performance burden it puts on connections. The OASIS Provisioning Services Technical Committee has tried to improve SPML in...