How To Evaluate New Software in 5 Steps

By: Bandita Awasthi on June 25, 2020

When it comes to business software, it’s not about whether you need it—it’s about which software you need. And when there are hundreds of possible choices, answering “which” gets really tough.

Stacy Caprio, founder of Growth Marketing, went with a popular option to manage email marketing but later realized it wasn’t financially sustainable. As her email subscriber list grew beyond 2,000 members, the pricing plan became too expensive and she eventually had to discontinue the tool.

Examples such as these are abundant among business organizations but they are avoidable. Thorough software research and evaluation can help you stray clear of software selection mistakes that can lead to costly consequences such as switching vendors or losing customers.

To carefully evaluate a solution, you need to craft a detailed evaluation strategy. This strategy should include various aspects such as the tool’s technical capabilities, performance of the vendor, and the total ownership cost. You’ll also need to factor in your business’s unique processes and requirements.

Since building such a strategy from scratch may seem like much effort, we’ve created a blueprint of this strategy for you.


1. Conduct needs analysis: put the 'why' before the 'how'

You can’t evaluate a software tool if you don’t identify the target outcome of using that software. To know how good a tool is, you need to first identify the key pain points you’re hoping to fix with it. For example, before evaluating an ecommerce system, you may want to ask yourself:

  • Are we missing out on higher sales by not having our retail store online?

  • Are our customers dissatisfied with the current user experience (UX)?

  • Are the in-house server maintenance costs becoming too much? Will subscribing to a cloud-based tool make more sense?

Understanding the “why” will make it easier to identify “how” to eliminate the irrelevant options. If you wish to offer a better UX to your customers, you can look for tools that offer a gallery of predesigned templates, a compelling shopping cart module, and a smooth checkout experience.

FAQ: What are the key things to consider before beginning the evaluation process?

Answer: Some key things to consider before evaluating a tool are as follows:

  • Who will head the evaluation process?

  • What are the budget requirements?

  • What all approvals are needed to get started?

  • What is the timeline for the evaluation?

The idea is to establish a workflow, get approvals, and divide responsibilities.

2. Identify 'must-have' features: create your own yardstick

After clearly understanding the pain points the tool will address, jot down a list of features that might help cater to those needs. In addition to the “must-have” features, also create a list of the “nice-to-have” features.

If we take the ecommerce software example, a compelling image gallery and editor, checkout system, and shopping cart module might be your “must-have” features, whereas an automatically adjusting webpage color scheme could be a “nice-to-have” feature.

Once you’re clear about the features you need, it’s time to start shortlisting the tools that meet your feature requirements. However, sorting through the numerous tools on the market to come up with an initial shortlist can be daunting.

For this, you can refer to research on top-ranking tools—our FrontRunners is one such report. You can also refer to third-party websites, such as ours, that list products based on user ratings.

It’s okay to dedicate a fair amount of time to this exercise as it lays the foundation for the next steps. Your focus should be on trying to make the list as exhaustive as possible.

FAQ: What is a software scorecard and how can it help me?

Answer: A scorecard can help you quantify a tool’s capabilities and compare it with other options. But it requires doing some math!

Create a list of your must-have and nice-to-have features and assign weights (i.e., a numeric value) to each based on its relevance to your process. The more relevant the feature, the greater the weight. The collective weight of all the features should sum up to 100.

Then score each tool on its individual features, on a scale of 1 to 5, and use this formula to generate a weighted score:

(Weight / 5) x Score = Weighted score

For example, if the tool scores 2 on Feature 1 and has a weight of 25 then the weighted score will be (25/5) x 2 = 10.

Once you’ve arrived at the weighted score for each feature, add them to get the final score. Do this for each tool to see how the final scores compare.

3. Select a vendor: pick a lasting partnership

Selecting the right vendor is just as important as selecting a tool with robust technical capabilities. Since software purchases are usually a long-term commitment, it’s best to take some time to thoroughly study the vendor. Here are some criteria on which you can evaluate the vendor:

  • Deployment options: Deployment options vary across vendors, such as on-premise, SaaS, and cloud ERP. Each deployment option varies in terms of data storage, maintenance, support, updates, etc. You’ll need to factor in these peculiarities before deciding on a particular vendor.

  • Customer support: The options for customer support vary across vendors. These may include email, phone, or chat support, whereas some vendors offer support only through tickets. Moreover, support can be freely available or paid.

  • Additional training: Based on the complexity of the tool, your team may require additional training before using the tool. Some vendors charge for these trainings while others offer free training or educational materials (such as how-to guides and video tutorials).

FAQ: How can I do effective vendor research?

Answer: There are many ways of doing vendor research. You can visit their website, read case studies, and refer to third-party websites to read unbiased vendor reviews. If you are aware of someone who uses the product, you can ask for their feedback on the vendor.

Additionally, you can refer to news sources to check if the vendor has been involved in any data security breaches or overcharging.

4. Calculate the total cost of ownership: factor in the upfront and hidden costs

Now that you’ve shortlisted the tools that are most relevant to your needs, it’s time to evaluate their total cost of ownership (TCO). Gartner defines the TCO for IT as the cost of “hardware and software acquisition, management and support, communications, end-user expenses, and the opportunity cost of downtime, training, and other productivity losses.”

Let’s see how you can calculate this:

  • Project initiation costs: If you’re replacing a legacy system, you’ll need to include the cost of retiring the legacy system or keeping it operational until it’s replaced.

  • Initial setup costs: This refers to the cost involved in setting up the tool and includes expenses incurred on hardware and software acquisition and implementation.

  • Ongoing costs: This cost involves the cost of maintaining and operating the tool year-upon-year. It includes expenses such as server maintenance fees, network equipment maintenance fees, annual upgrade costs, technical support staff, and administrative training.

  • Projected costs (5 years): Based on the initial setup and ongoing costs, generate an estimate for the next five years to find the total cost you’ll incur. You should also include the costs of depreciation and amortization in this section. All of these costs combined will give you an estimate of the tool’s TCO.

FAQ: How does the TCO vary for on-premise and SaaS deployments?

Answer: The TCO will vary based on the initial setup—on-premise deployment usually involves higher upfront investment than SaaS. It will also vary based on licensing and subscription, maintenance and support, data migration, and hardware costs. To get an idea about how the cost might vary, check out our on-premise and SaaS TCO calculator.

5. Speak with the stakeholders: decide as a team

It’s essential to include the opinion of relevant people as they have a direct stake in the purchase: they may be the end-users of the tool or the process heads responsible for ensuring efficiency gains. Either way, the more perspectives you have, the better.

Here are some ways to ensure everyone’s voice is heard:

  • Conduct a survey: With several stakeholders involved, it may be difficult to get hold of everyone. One easy yet effective way of gaining feedback in such cases is to conduct a survey. Create a survey using Google Forms or any survey builder tool and analyze the key findings.

  • Brainstorm as a group: If it works for everyone, find some time to brainstorm as a group. This approach ensures doubts are put to rest and you may get some valuable feedback in the back-and-forth of questions and answers.

  • Get one-on-one feedback: With senior stakeholders that are important to the decision-making but tight strung on time, consider having one-on-one sessions.

FAQ: How can I conduct a survey effectively?

Answer: To conduct a survey, identify the key areas you need to address and consider these to be the key buckets. Come up with the relevant questions and brainstorm around these buckets. The optimal number of questions is between 10 and 15.

Once the questions are ready, identify the means of conducting the survey.

Don’t hesitate to seek support

We hope you found the software evaluation steps helpful as a blueprint. If you’re still unclear on any aspect or feel you need additional support, reach out to our advisors for a free, no-obligation consultation.