Insider Tips on Buying a SCADA System

Enhance the process by understanding the vendor’s point of view.

by
Mike Walden, Control Systems International, Inc.
913-599-5010 • mwalden@csiks.com
This paper is based on one presented at an ENTELEC conference or seminar sponsored by the Energy Telecommunications and Electrical Association, Inc.

Introduction

Background and Purpose

As you know, replacing a SCADA system is a big deal. It is a high visibility project within the
company and may even attract the attention of the trade and general press. Just like everything
else about maintaining a process, replacing a SCADA system can have significant impact in any
number of areas, especially on operations.

The irony is that despite the importance of a SCADA replacement project, most companies don’t
have all that much experience doing it. Most companies replace a SCADA system every 10 to
15 years. Typically, it can take two to five years to complete the system. And with all of the
recent consolidation in some industries, it is highly probable that most of the people who
performed the last upgrade are not available to perform the latest one.

As a consequence, the tendency is often to treat a SCADA replacement project just like any other
computer hardware or software application replacement process. After all, you are purchasing
hardware, software, and services. It should be possible to just scale up the process. Besides, your
company may have purchasing procedures that cover this process, and you are obliged to follow
them. But replacing a whole SCADA system typically requires a whole different approach than
replacing any of its components. The process for acquiring a new server or a new operating
system might be efficient. But that same process applied to an entire SCADA system might
actually impose undesirable obstacles and reduce your overall satisfaction with the new system.

Many companies have specific procedures for replacing a SCADA system. Those procedures
worked the last time they acquired a SCADA system. But, that was a long time ago. Many
companies find that the technology has changed since then, the vendors have changed, and the
old procurement process is no longer as efficient as it might be.

This paper will:

• Identify the most costly and inefficient purchasing methods commonly used to acquire a
SCADA system, and …

• Identify simpler, more efficient, and more productive methods for specifying, selecting,
and purchasing a SCADA system.

Insider Tips on Buying a SCADA System Page 1

As the paper’s title suggests, these recommendations are made from the point of view of the
SCADA vendor.

Why Care about the SCADA Vendor’s Point of View

The problems and recommendations discussed in this paper are not about making life easier for
SCADA vendors. In fact, some of these recommendations place more of a burden on the
SCADA vendor. But experience has taught SCADA vendors that successful projects are far
more likely when both the vendor and the customer work together to set expectations and
define how best to achieve them.

Actually, SCADA vendors have a level of experience and a perspective about the SCADA
replacement process that end-user companies do not have.

An end-user company might purchase a SCADA system once every 10 or 15 years. But SCADA
vendors are involved with many SCADA system purchases each year and might actually
participate in proposals and bids several times a month. Vendors are in a position to see what
works and what doesn’t work – from specification preparation to proposal and bidding rules.
SCADA vendors know that the procedures for selecting a SCADA system can affect the cost
and viability of that system in the near term and for years to come.

After many years of experience, vendors get good at predicting the future success of a project –
whether or not they win the contract. Very often, what a vendor sees in the specification, the
bidding rules, and the bid documents tells them what they need to know. And more often that
not, what they don’t see in those documents tells even more.

Who is CSI and Why Are We Qualified to Talk about This?

Control Systems International is an automation company that specializes in upgrading existing

SCADA systems.

Over the last 35 years, CSI has implemented more than 1,400 automation projects worldwide.
In the last eight years, about 65% of CSI’s business has come from upgrading SCADA systems.
CSI has excelled in this niche market by developing very effective software tools, procedures,
technologies, and instincts that allow us to upgrade systems faster, more efficiently, and with
lower risk to our customers.

This paper intends to share some of that experience with you.

To a great degree, the success or failure of a SCADA upgrade project is cast in the earliest
stages of that project. Those early stages include:

• Organization the Process

• Analyzing Needs

• Defining Requirements

• Evaluating Prospective Vendors

We will consider key issues in each of these stages – key issues that impact the project’s
success or failure.

Insider Tips on Buying a SCADA System Page 2

Organizing the Process
One of the most important and difficult decisions is often the very first decision: do you upgrade
with internal resources or hire an outside consultant?

Some of the arguments for each side in this debate are obvious. You know what you want better
than anyone, but a consultant may have more experience in the actual process of acquiring a
SCADA system.

There’s a lot to be said in favor of both approaches. Neither is an inherently good or bad
approach. We summarize below the key issues and how they affect both of these approaches.

Have a Clear Purchasing and Evaluation Plan

Whether you go it alone or hire a consultant, a clear purchasing and evaluation plan needs to be
in place.

Consultants are usually technically proficient but can be organizationally challenged. They might
know all the latest buzzwords and jargon. But their last actual involvement in the procurement
process may be dated. If a consultant does not bring to the table a clear procurement process
based on recent experience, his or her excellent technical abilities might serve only to waste
time and frustrate the project team.

Make sure your consultant has a documented plan for identifying your needs, for documenting
those needs, and for identifying the best vendors to approach for proposals and/or bids.
Whether you hire a consultant or do this yourself, the purchasing and evaluation plan needs to
include a time line for at least the following major tasks:

• Requirements development

• Procurement process

• Bidder selection process

• Bidder evaluation process

• Vendor award process

• Configuration

• Factory acceptance test

• Start-up

The plan should also indicate what percentage of each stage will be performed in-house and
what percentage will be performed by the consultant, vendors, or others.

This is your road map. As the philosopher once said, if you don’t know where you’re going, how
will you know when you’ve arrived?

Insider Tips on Buying a SCADA System Page 3

Politics and Lines of Authority

One ineffective technique is to appoint a project team to oversee the upgrade process and give
everyone on the team equal authority. This is a recipe for inaction, schedule delays, and,
eventually, cost overruns.

Actually, we believe in having a project team that participates in the upgrade process from the
very beginning stages. This team should include all of the major stakeholders in the SCADA
system, including field engineers, technicians, analysts, operations personnel, IT people, and
management.

Certainly, the field and operations people will not take kindly to a system that management
imposes on them. So for political reasons, it is important to make sure that their experience,
needs, knowledge, and points of view are represented from the beginning. Besides being good
politics, the field and operations people are likely to add the most value to the project; they know
in detail what will and won’t work in daily practice.

By the same token, it doesn’t make sense to turn the entire process over to the technical staff. If
everyone does not keep a sharp eye on bottom-line goals – regardless of the technical issues
involved – then the process will not succeed.

Once you have a project team, make sure that a single person is in charge. That person must
have the authority to make decisions and to ensure that the system meets both the company’s
technical and business goals – especially the company’s business goals. When one person has
his or her reputation on the line, the project succeeds.

The same goes for a consultant. If you hire one and then ignore his advice or undercut all his
proposals, what good is he? The consultant must report to someone high enough up the
management chain to ensure that the consultant’s proposals can be objectively evaluated.

An Alternative Approach

Before we leave this first stage of the process, I’d like to discuss an alternative approach to the
usual way of acquiring a SCADA system.

The usual approach is to do things by the numbers. In other words, develop a specification that
reflects what you’ve got now or that reflects what you want. Make that specification so detailed
that it leaves nothing to the imagination. Then send the specification to a number of vendors.
Then evaluate the vendors’ responses by the numbers: The vendor with the fewest exceptions
and/or lowest bid wins.

However, the specification on which those bids are based is usually written to ensure a
reasonably broad number of competitive bids. That means each vendor might have the best
technology and lowest price in one area but not in most areas that you ask them to bid on. The
winner isn’t necessarily the best match, but rather the least objectionable match.

We have been impressed by an alternative approach that we have witnessed among some
SCADA users. We didn’t invent this approach, but we are impressed by the successes that the
approach has generated. It goes something like this.

Insider Tips on Buying a SCADA System Page 4

You appoint a project team much as we outlined previously. But rather than starting by
identifying your company’s needs and then writing a specification, this team goes on a fact-
finding tour. They use vendors, trade shows, other companies in their industry, and other
sources to educate themselves. They learn about the latest technology and SCADA procedures
that have become available since the last time they bought a SCADA system.

The concept here is simple: What you can do is limited by what you think is possible.

One organization spent two years researching the latest technology and potential vendors. They
visited numerous vendor corporate offices and customer sites. Basically, they used vendors as
free instructors.

This organization realized that they needed to find out what was technically possible in a
replacement SCADA system. They didn’t want to miss any opportunities. One of the things they
discovered is that they didn’t need to replace all their equipment just to upgrade to the latest,
cutting-edge SCADA technology. They didn’t know that before they embarked on their
educational efforts. Finding that out before they wrote their specification saved them a large
amount of money.

In the end, this organization says that they not only got the lowest bid, they also got the best
match in a vendor.

If you don’t know all that is technically possible to accomplish in a new SCADA system, then
you are likely to miss opportunities that would otherwise put you at a competitive advantage.

Once you have determined what is possible, you can write a specification that realistically ties
your needs to available technology. That way you successfully avoid pie-in-the-sky
requirements that are either technically impossible or impossibly expensive. You also avoid
writing a specification that puts you no further ahead competitively than you are now – or were
ten years ago when you originally bought the system.

However, even with all that education, you may still end up with a winning vendor whose bid is
the best compromise, not the best match for your needs. Here’s how some companies avoid
that pitfall.

Some companies decide not to engage in the traditional specification and vendor selection
process, as we know it. Instead of writing a spec, soliciting bids, and choosing the lowest bidder,
they look for a SCADA system partner. These potential partners are, most likely, the same
vendors that would appear on a bid list. But instead of evaluating them by the numbers, you
evaluate them by more subjective criteria: their depth of experience, the judgment of their other
customers, their experience with the challenges you face, their ability to communicate with you,
their technology, and their general estimate of what it might cost them to develop a system for
you.

Once you choose such a partner, you work with them to develop a detailed specification that not
only meets your needs but also takes full advantage of their resources. The more you and your
partner know about each other, the more opportunities you see for each other.

At first glance, you might think that this approach would be more expensive than the traditional
approach because it eliminates some of the competitive aspects of the process. However, in
practice, the companies that we know who have used this approach actually saved money.
Typically, the actual expenses came in at less than the company’s original budget. Why?
Because both sides worked to maximize each other’s strengths. That is not necessarily the case
in the more traditional, more adversarial vendor/customer relationship that is the hallmark of the
traditional bidding process.

Insider Tips on Buying a SCADA System Page 5

Analyzing Needs
Now let’s consider the next stage, needs analysis. Why do you want to replace your SCADA
system?

The absolute worst answer is “because it’s time.” Maybe your system’s components have
reached end-of-life and are hard to repair, but they still work. Maybe they no longer represent
cutting-edge technology. Maybe you’re tired of your competition whipping out photos of their
shiny new SCADA system while you are embarrassed to show worn-out photos of your ancient,
toothless SCADA system.

These are not usually compelling reasons to upgrade.

In fact, the only good reason to upgrade is because it achieves one or more tangible business
goals. The exact goals will vary from company to company. What’s important is that those goals
be identified and quantified, and that they become the most significant criteria in selecting a
replacement SCADA system.

Defining Requirements
Once you have defined the goals, then you need to specify what it will take to achieve those
goals. Will a newer version of your current SCADA system achieve those goals? Do you need to
replace all your hardware to achieve those goals? What new technologies can be applied to
achieve the company’s business goals?

Typically, the answers to these and other questions end up in a specification that is ultimately
put out for bid.

In preparing the specification and other bid documents, many companies make one or both of
the following mistakes:

• They define project requirements in vague terms that cannot be measured, or …

• They define project requirements in so much detail that few bidders can cost-effectively
fulfill those requirements.

Let’s look at each of these a little closer.

Overly Vague Bid Documents Lead to Overly Vague Bids

Let’s see now, we want to replace our SCADA system in order to accomplish the following
goals:
• Improve usability.
• Maximize flexibility.
• Provide a platform for future expansion.

Insider Tips on Buying a SCADA System Page 6

Improve the usability of what? Do you want your HMI icons to be prettier or do you want your
operators to use fewer mouse clicks to close a valve? And exactly how do you measure
improvement? Efficiency experts say that any change in the work environment – even a
negative change – causes at least a temporary boost in productivity. It’s called the Hawthorne
effect. So, according to the Hawthorne effect, it’s possible that we could improve usability by
painting the I/O racks pink. But I doubt that’s what the author of “improve usability” had in mind.
If you want to improve usability, you at least need to state what constitutes an improvement, not
to mention what you expect to improve.
The same is true of maximizing flexibility. The flexibility of what? And what constitutes flexibility?
And exactly how much is “maximum”? When my daughter asks for maximum flexibility in her
curfew, I guarantee you that her definition of the words “maximum” and “flexibility” are vastly
different from mine. Unless those words are defined, they can only lead to confusion,
miscommunication, and added costs.
Do we need to discuss “provide a platform for future expansion”? It suffers from the same lack
of specificity as the other vague goals.
Vendors think it’s very important to know what your goals are. It helps vendors tailor proposals
and bids to meet those goals.
What’s tragic is that even those bid documents that clearly enumerate goals often go too far.
Overly Detailed Bid Documents Lead to Overly Costly Projects
Let’s say you’ve written a tight, comprehensive specification that states exactly what you’d like
to purchase. You then proceed to tell bidders how to accomplish your goals using the tools you
have dictated. SCADA vendors will jump at the chance to bid this type of project because we
can submit the lowest price to win the job and all the risk is back on you.
This is a big mistake. You can’t think of everything in the initial stages of a project, yet this
approach assumes you have. Right from the beginning you’ve either closed the door on
improving the SCADA system, or you have made doing so very costly. In the end, the customer
who takes this approach ends up paying more for his project than it should cost. Or even worse,
the system does not perform the way the company expected it to perform.
Well, that’s exactly what many companies do. And they miss out on a tremendous opportunity –
the opportunity to get lots of free advice from willing vendors.
We call these two types of specifications performance vs. procedural.
• The performance specification or request for proposal states exactly what the company
wants to accomplish.
• The procedural version states exactly how the company expects to accomplish their
goals. The procedural version is often loaded with lots of technical specifications for
networks, timeouts, software, RTUs, PLCs, protocols, radios, etc. Very often, procedural
versions are written by consultants who lock their clients into very narrow options.

RFQs and RFPs Should be Written, Not Collated

You’ve heard the old saying, “A camel is a horse designed by a committee.” In far too many
cases, RFQs and RFPs look amazingly like camels.

Insider Tips on Buying a SCADA System Page 7

There are often contradictions between the specification and the compliance table. Sometimes
the commercial requirements conflict with the technical requirements. Sometimes the schedule
and payment milestones negatively affect execution and performance. Many specs and bid
documents look and read like what they are: different documents pulled together with no
coordination among sections.

Of course, sometimes you end up with a camel even without the involvement of a committee.
For example, it is quite common for a consultant to take the following approach to writing a
specification:

1. Start with a previously written specification, perhaps a good one.

2. Solicit sample specifications from several SCADA vendors.

3. Choose the best features of each system and compile them into your specification.

This is the “Greatest Hits” style of specification. It’s certainly easier than writing a specification
that addresses your needs.

Another problem is that many specifications contain varying degrees of detail. Perhaps the
committee member assigned to develop the architecture supplies detailed drawings and
itemizes every workstation, every controller, every server, and every communications hub. But
another committee member fulfills his or her assignment by specifying nothing more than the
phrase “include a simulator”.

Such discrepancies make vendors wonder whether or not we are getting the whole story. If we
think that you are under- or over-specifying your system, we tend to under- or over-estimate our
costs. If we under-estimate them, you’ll end up paying more. If we over-estimate them, you’ll
end up paying more.

Bid Sections Should be Evaluated for Relevance and Usefulness

One of the more problematic sections is the compliance table. The compliance table can be a
useful tool. But before you include one in your bid, you need to ask yourself what you expect to
learn from the compliance table. If every bidder indicates that they are compliant on every point
– which is very often the case – what does that tell you? Is that credible? Does it mean that all
bidders are equal?

Some vendors consider taking exceptions to a specification as a negative, not an opportunity to

show how they are different or even better. As a consequence, they bend over backwards to
avoid taking any exceptions.

How will you evaluate exceptions? Will you consider the impact of each exception on the
project, or will you just add up all the compliant check marks and play it by the numbers?

A more useful approach would be to provide space for the vendor to describe an exception and
how it might impact the project. Minimally, the compliance table should allow the vendor to flag
an exception for further discussion with the customer so that the customer can understand the
relevance of the exception to the project.

This suggestion also applies to other parts of the bid. For each section of your bid document,
ask yourself how you plan to evaluate that section, its significance within the evaluation, and
whether or not it contributes any tangible value to your evaluation. You should also determine
whether or not the section can be evaluated, and if so, whether or not you can easily compare
vendor answers.
Insider Tips on Buying a SCADA System Page 8
Here’s an example of what I’m talking about. Many bid documents request a company history.
As a vendor, we are happy to supply one. But what are you looking for in such a section? Are
you looking for an older company or a newer one? Are you looking for an innovator or a
company that can execute traditional technology? If you ask for something as nebulous as a
company history and you’re looking for something specific, there’s no guarantee that you’ll get a
response in a form that tells you what you need to know. If you’re looking for something specific,
ask specific questions.

If you just want to learn about the company’s background, that’s fine. But if that background is
supposed to tell you something significant and you have not figured out what the significance is
before you send out the bid documents, consider getting rid of the section. Otherwise, it just
makes more work for you during the evaluation.

Evaluating Prospective Vendors

Evaluating vendors and their responses to your questions should be a combination of art and
science, a combination of corporate chemistry and bottom-line common sense. Too many
companies compile and evaluate bids by the numbers. I’m not talking about money. SCADA
users in the past have worked very hard to try to normalize vendor bids. But in many cases the
criteria and point value assignments are not always developed to provide an objective
evaluation. This is always a hard task, but it must be done.

It’s not so easy to compare less quantifiable factors like customer references and project
methodology. It’s especially difficult calculating the effectiveness of a long-term relationship with
the vendor. But if your project is to succeed, those factors must be at the top of the list and
given due consideration.

Identifying Potential Vendors

One of the first challenges you must face is identifying a list of bidders.

It is an urban legend to think that the more competitive you can make the bidding, the lower the
long-term cost of ownership will be. So why not send an RFQ to every system integrator you
can find?

First of all, having a lot of bidders does not ensure a lower price. That’s because few vendors
are qualified to bid on – much less execute – your project. They could all be highly competent.
But they may not all be competent to execute a project like yours. So you have to narrow the list
down. If you don’t, you will get a lot of unrealistically low and high bids from vendors who don’t
understand what you need. It takes time to eliminate all the unqualified bids.

There are several ways to narrow the list. You could ask for a proposal in which you outline your
project and ask potential bidders to propose how they might execute such a project. You could
also ask them to put together a least-cost estimate on a generic architecture. This would give
you some good ideas as well as allow you to examine how well different vendors respond to
your criteria. Of course, you would also ask for specific company details, references, etc. to help
further narrow the list.

Insider Tips on Buying a SCADA System Page 9

If your project is especially large, you should consider examining each potential vendor’s project
execution methodology. Many engineers fly by the seat of their pants. They rely on their
experience, ingenuity, instinct, and knowledge to basically make it up as they go along. That
might be acceptable for a small project. But it can be a disaster for a large project that involves
multiple engineers and requires clearly defined lines of communication between customer and
vendor along with exacting accountability. A documented project execution methodology can
help ensure that all members of the team understand their role, understand company
procedures, and understand the best way to accomplish major milestones. This might consist of
a set of standard operating procedures, work instructions, or other tools that allow the vendor to
manage projects to a successful conclusion.

At some point, you will need to test that project execution methodology. How can you do that
short of actually hiring the vendor? There are several ways. Ask each vendor staff member that
you meet where they fit into the project lifecycle. Do they seem to act more like independent
contractors or more like a team with a common sense of mission?

Next, talk to the vendor’s customers. Do their customers see tangible evidence that the vendor
has a project execution plan? Does the vendor actually follow the plan, or is it just to impress
prospects? Does the vendor provide written reports on a regular schedule? Are key vendor
contacts available when needed? Can you talk to them and get straight answers from them or
do they constantly defer to their supervisor? How is the after-project support? Go on site visits
and see the installed systems first hand.

Choosing a SCADA system vendor is risky. You need to find out whether the vendors on your
final bid list add to that risk or reduce that risk.

If you end up making a final choice by the numbers, please make sure that the list of vendors
you have to choose from are vetted first. Make sure that final list includes vendors that your gut
tells you can do a good job. And make sure your gut is well informed.

Develop a Selection Process

Very often we see RFQs that include the company’s selection criteria. While that’s very good for
us vendors to know, we are amazed to see how complex that criteria can be. Very often its
complexity rivals the complexity of the specification. It seems that scoring such bids will take a
lot more work and time than it took to create the bid.

We have seen huge matrixes with literally hundreds of criteria listed – some with significance
weighting and some without. Learning how to score such a bid can take hours. You have to
wonder how consistent scoring can be with such complex criteria. Most of it still boils down to a
judgment call, and those judgments can’t possibly be consistent with so many criteria to consider.

Another important factor is who you get to help evaluate the bids. Make sure that each
significant stakeholder is represented, such as field engineering, operations, IT, and
management. First of all, it’s good politics. But even more important, each of those disciplines
brings a perspective to the evaluation that is crucial to making the right choice. Management
might like the low bidder, but field engineering and operations might find flaws in that bidder’s
technical proposal. By the same token, operations might like the technology in one bidder’s
proposal, but management might find that proposal does not support the company’s goals.

And as I stated before, make sure someone is in charge. We think that the company’s best
interests are served if that someone is from management. They can best sell the project team’s
decision to senior management. But more important, they are best qualified to ensure that the
winning bid meets the most important criteria of all – adherence to the company’s business goals.

Insider Tips on Buying a SCADA System Page 10

Summary
Of course, my colleagues and I have many other ideas for making the bidding process more
effective. I invite you to visit our web site (ucos.com) to learn more about us.

To summarize:

• Develop a clear purchasing and evaluation plan. If you hire a consultant, make sure the
consultant has such a plan.

• Put one person in charge. This project should have an impact on his or her career.
Involve the company’s major stakeholders in the research, bidding, and bidder
evaluation process. These stakeholders can include field engineering, operations, IT,
and management.

• Instead of writing a specification and sending it out for bid, consider researching the
state-of-the-art in SCADA system technology first. That way, when it comes time to write
a specification, you’ll know what’s possible and how you can take full advantage of the
latest technology in achieving your company’s goals.

• Instead of choosing a vendor by the numbers, consider choosing a partner whose

technology, skills, and style match your own. Then work with that partner to develop the
best possible system that the two of you can muster. This approach requires you to
research potential partners quite thoroughly and then trust yourself to choose someone
you think you can work with. You might just find that this approach not only costs you
less but also results in a system that works better.

• Analyze your needs based on achieving company goals. You don’t need new
technology. You may, however, need to lower your cost of operations. New technology
is one way to accomplish that goal. However, new technology is not the goal.

• When defining your system’s requirements, do not use vague phrases to specify what
you want. That only results in vague bids. On the other hand, do not specify what you
want so narrowly and in so much detail that you cut off a tremendous source of free
input: your vendors’ best efforts at designing a system that achieves your business goals.

• Make sure the criteria you use to select a vendor provide you with relevant and useful
information about prospective vendors. If you don’t know why you are asking a vendor to
do something, then don’t ask. Ask them for information that you can use.

• Do not base your vendor selection primarily on written bids. Of equal or greater
importance is what the vendors’ customers think of them, as well as your gut feeling as
to how you and the vendor can work together.

Remember, the way that your bid documents are written and organized has a significant impact
on everything from the price to the system’s ultimate viability. No matter how good the answers
are, you must still ask the right questions in order to make the right decision.

Mike Walden
Control Systems International, Inc.
Lenexa, Kansas 66214 USA
Voice 913-599-5010 • Fax 913-599-5013
mwalden@csiks.com - www.ucos.com

Insider Tips on Buying a SCADA System Page 11