How to check a software vendor’s references

These questions are geared towards client/case management systems and are probably more numerous than would be reasonable or practical to ask in a real-world scenario, but may be useful when used selectively in other contexts.

Italics explain why a particular question is being asked.

Organisational questions about the referee

This information will help your organisation understand the referee’s organisation and situation and whether it aligns with your organisation’s needs. Possible questions include:

• What does your organisation do? How many staff members do you have? How many clients do you provide services to? How many and what services do you provide?
• How long have you been with the organisation? What do you do within the organisation? How do you use the system? (The view from an IT person will be different and have a different weight than from a CEO.)
• What system did your organisation use before? (If it was a ‘poor’ system previously, any average system will seem wonderful now.)
• Why did your organisation acquire this system? What was your organisation hoping to achieve?
• How long has your organisation been using the system? What is your organisation using the system for? How many users does your organisation have?

Quality of the software

This information should help you understand more about the software that your organisation is considering. Possible questions include:

• What version of the software does your organisation have now? (If different from what you are being offered, problems that the referee organisation may have experienced may have been addressed in the interim.)
• Is the system reliable? Does it do what your organisation expected it to do?
• Are there any workarounds that the users need to use that they did not know about beforehand? Is there any plan to address these?
• Are the configurable parts of the system easy to modify to meet your organisation’s particular requirements?

Implementation and data migration

This will give you an indication of:

• how accurately the vendor understands a client’s particular requirements
• how accurately the vendor estimates the work required to meet the client’s needs
• the vendor’s team’s capability to scope and project manage an implementation, delivering to a timeline and a budget
• the amount of work the client needed to undertake - internal project management, user acceptance testing, user training, etc. For some systems, this can be a significant time and cost.

Much of this will also be applicable for any subsequent additions, changes or enhancements that may be requested.

Possible questions to ask include:

• How accurate was the proposal with respect to time and cost? If the proposal was inaccurate, what was the main cause?
• What was the implementation process like?
• Was there a written implementation plan with responsibilities, milestones and dates?
• What were your resource requirements for the implementation (e.g. time, people, etc)?
• Was the vendor’s implementation team on-site or available during the first reporting cycle? Did everything go as planned?
• Was the quality and amount of the training satisfactory?
• Did the vendor migrate data for you? How accurate was this process? What were the costs associated with data migration?

Quality of the vendor's customer support

These questions give you a sense of the ongoing support that is being provided. Questions include:

• What is your relationship with the vendor like?
• How knowledgeable are the vendor’s customer support people?
• What are the response times for help or issue resolution and are you satisfied with these?
• How do the vendor’s customer support people treat your organisation’s people?
• How does the vendor charge for support (e.g. fee for every question)?

Ongoing IT resources needed to support the system

It is important to have a good understanding of the amount and type of resource you will need to support your new system. This will help you understand how to get the best out of the system. Questions include:

• How much time does it take your organisation to manage the system?
• What level of internal IT expertise is required to manage and maintain the system?

Flexibility and future use

This is to determine how flexible the system is. How has the system been able to adapt to the referee organisation’s changing needs and the vendor’s response to those changes.

• Have you ever requested any enhancements?
• If so, did you get them? Were the estimates for time and cost accurate? How long did it take to get the enhancements?

Vendor's commitment to your success

Systems involve a significant investment and these partnerships are generally long ones. These questions will help you to develop a sense of the values of the vendor organisation. Questions include:

• Is the vendor fair and reasonable?
• Do you enjoy dealing with the vendor?

Closing question

• If you could do it again with what you know now, would you make the same decision?