CLM buying
CLM Reference Check Questions: What to Ask a Vendor's Customers Before You Sign
Reference calls are the cheapest due diligence in a CLM purchase and the most often wasted. A buyer asks the vendor for "a couple of references", gets two enthusiastic customers who agreed in advance to take calls, asks whether they like the product, and hears that they do. Twenty minutes, no new information. A useful reference check works differently. You ask for a specific kind of customer, you ask questions that cannot be answered with "yes, it is great", you listen for hesitation rather than praise, and you find at least one customer the vendor did not choose. This page sets out how. ## Ask for the reference you need, not the one on offer Every vendor has a short list of champions: customers who had a smooth rollout, like the account team, and are happy to speak. Talking to one is fine. Talking only to them tells you the best case. Send the vendor a written request describing the reference you want: - Similar size: a legal team of roughly your headcount, and a contract volume within a factor of two of yours.
- Similar use case: if you mostly review counterparty paper, a customer who does the same; if you mostly generate sales contracts, one who does that.
- Live long enough: at least one renewal behind them, so they can speak to the second-year price and support.
- Similar setup: a customer using the same plan you are considering, because features and support often differ by tier.
- One who left, or nearly did: vendors rarely offer this, but asking is informative. Some will connect you with a customer who had problems and stayed; that call is often the most useful. If the vendor cannot find anyone matching your size and use case, that is a finding in itself. ## Homework that makes thirty minutes enough Before each call, spend fifteen minutes on the reference's company. Look at its size, industry and the kind of contracts it is likely to sign; a SaaS company selling to enterprises lives on counterparty paper, a staffing firm lives on its own templates. Check whether the person you will speak to was there when the tool was bought, because someone who inherited a system speaks about it differently from someone who chose it. Then pick the eight or ten questions from the list below that matter most for your purchase, rather than racing through all of them. A reference who feels rushed gives short answers. Agree internally on what would change your mind. If a reference tells you renewals rose sharply, or that the AI review is mostly ignored by their lawyers, does that stop the purchase or start a negotiation? Deciding this in advance stops a single persuasive call from swinging the decision either way. ## Set the call up so people speak freely Ask for thirty minutes with the customer alone. Say at the start that you will not quote them to the vendor, and keep that promise. Have one person ask questions and another take notes, and bring the person on your side who will actually use the tool, not only the buyer. ## The questions, grouped by what you are trying to learn ### From signature to the first live contract 1. How long from signature to the first contract handled in the system, and what was the vendor's estimate at the start?
- Who did the setup work: the vendor, a partner, or your own team? How many of your people's hours did it take?
- What was harder to set up than the demo suggested?
- Which of your existing contracts did you migrate, and how clean was the extracted data afterwards? ### Whether licensed users actually log in 5. Of the people who have licences, how many use it every week?
- Which team or person still works around it, and why?
- What did you have to change in your own process to make it stick?
- If the tool disappeared tomorrow, what would your team miss first? ### Support after the sales team steps back 9. Describe the last time you raised a support issue. How long until a useful reply, and from whom?
- Has your account manager changed? How many times?
- Is the help you get now the same as the help you got during the sale? ### How the AI output holds up on real drafts 12. How often do reviewers change or reject what the AI suggests on a typical contract?
- Where does the AI get things wrong in a way you have learned to check?
- Does it handle the other side's paper as well as your own templates?
- Has the quality changed over time, better or worse, and did the vendor tell you when the underlying model changed? ### Price at signature versus price at renewal 16. Was the first-year price close to what you budgeted once implementation and add-ons were included?
- What happened at renewal: price change, seat minimum change, features moved to another plan?
- Were there costs you did not expect, such as storage limits, AI usage caps, integration fees or extra charges for e-signatures? ### What the customer's security team found 19. Did the vendor's written answers on data handling hold up when your security team looked closely?
- Have you exported your data? How easy was it? ### Hindsight from the customer's side 21. What do you know now that you wish you had known before signing?
- What would you negotiate differently?
- Did you seriously consider another vendor? Why did you choose this one, and would you choose it again?
- Who else should I talk to? The last question matters more than it looks. Customers often know peers using the same tool, and that second-hand introduction is the easiest route to a reference the vendor did not select. ## Reading between the lines References rarely criticise a vendor directly, especially one they chose. The useful signals are quieter: | What you hear | What it often means | |---|---| | A long pause before "it went fine" | Implementation was harder than they want to say; ask what took longest | | "We mostly use it for storage now" | The drafting or review features did not stick | | "Our champion left, so..." | Adoption depended on one person; ask how many weekly users remain | | "Support is good once you reach the right person" | First-line support is slow or scripted | | "We are still on the original plan" | Possibly fine, or possibly they avoided a tier change because of price; ask | | Vague answers on AI accuracy | They have not measured it, or the reviewers still check everything | | Detailed, specific praise | Usually genuine; specificity is hard to fake | Write down what was not said as well as what was. If three references all avoid the renewal question, ask the vendor directly. ## Channels the vendor does not choose Balance vendor-arranged calls with sources the vendor does not control: - Peers in your network. A message to legal operations contacts asking "who uses this, and would they talk?" often finds a customer within a day. Communities such as CLOC, the Corporate Legal Operations Consortium, bring together legal operations professionals who compare tools openly.
- Review platforms. Gartner Peer Insights and Capterra both list reviews of contract lifecycle tools. Sort by most recent, read the critical ones, and filter for company size where the site allows it.
- Former employees and implementation partners. Partners who implement several CLM tools have seen rollouts fail and succeed; ask them which tool fits your size, and note whether they resell any of the options.
- Public statements. Conference talks and posts by legal operations leads describing a rollout are often franker than reference calls. On review sites, read for repeated patterns rather than averages. The FTC's rule on consumer reviews and testimonials, codified at 16 CFR Part 465, prohibits fake reviews, undisclosed reviews from company insiders, buying reviews conditioned on positive or negative sentiment, and suppressing negative reviews. That helps, but a lawful review still reflects one customer's situation, which may be nothing like yours. ## Putting the calls to work in the decision After each call, score four areas on a simple scale of green, amber or red: implementation, adoption, support and money. Then compare the pattern across all your references, including the ones you found yourself. One red is a question to put to the vendor. The same red from two independent sources is a negotiating point, such as a cap on renewal increases, a named support contact in the contract, or implementation milestones tied to payment. Reference findings also improve your pilot. If references say the AI struggles with a certain contract type, put two of those contracts in your test set. To compare what references told you about rollout time with a realistic plan, see the CLM implementation guide for US teams, and for turning reference findings into scored requirements, see our CLM RFP checklist for US buyers. ## Running this check on Adira Adira is a younger AI contract lifecycle vendor, and the reference approach above applies to it as much as to any established name. Its own measured result is that 18 law firms use it and that across those deployments contract review and due diligence that took 10 to 12 days manually came down to under 2 hours, with only 2 to 3 rechecks needed. The natural reference request, then, is a law firm among those users that resembles yours in size and practice area; ask Adira to arrange the call, and ask the firm questions 12 to 15 above about the AI output in particular. Two cautions specific to Adira. First, its references today are law firms, so an in-house team at a US company should press harder on the use-case match and lean more on its own pilot during the 7-day free trial. Second, its legal depth is greatest in Indian law, with other jurisdictions architected but to be checked contract type by contract type, so ask any reference which governing laws their contracts use. A useful question for a younger vendor is whether the reference had direct access to the people building the product, and whether their requests changed it. ## Questions to put back to the vendor afterwards Reference calls usually raise questions only the vendor can answer. Send them in writing and ask for written replies: - A reference mentioned a price increase at renewal. What cap will you put in our order form?
- References described setup taking longer than estimated. Which milestones will you commit to, and what happens if they slip?
- One reference said the AI struggles with a particular contract type. Can we test that type in our pilot?
- Our references were all on a different plan from the one we are buying. Which features or support levels differ? How a vendor responds to findings from its own customers is the last and often most honest piece of evidence you collect. A vendor that answers specifically and offers contract terms to back the answer is behaving the way you want it to behave after signature. ## A one-line test before you finish When the reference calls are done, try to complete this sentence: "Customers like us find that this tool is good at ___ and weak at ___." If you cannot fill the second blank, you have not finished checking.
Frequently asked questions
- How many CLM references should we call?
- Two or three, chosen to match your situation, is usually enough when combined with one or two references you find yourself. More calls from the vendor's list tend to repeat the same story; one independent conversation adds more than a fourth vendor-chosen one.
- Is it acceptable to ask a reference about pricing?
- You can ask about pricing structure and renewal experience without asking for their exact numbers. Most references will say whether the renewal increase was in line with what they were told, whether seat minimums rose, and whether features moved to a higher tier.
- Should the vendor be on the reference call?
- No. Ask politely for the call to be just you and the customer. Most vendors accept this. If a vendor insists on joining every reference call, weigh what the customer says accordingly.
- Can we trust review sites for CLM software?
- Use them for patterns rather than scores. Read the lowest-rated recent reviews and look for repeated complaints about implementation, support or AI output. Since October 2024 the FTC rule at 16 CFR Part 465 prohibits fake reviews, undisclosed insider reviews and buying reviews conditioned on sentiment, but it does not make every review representative of your use case.
- What if a new CLM vendor has few customers to offer as references?
- Ask for the closest match it has, and run a longer pilot instead of relying on references. A younger vendor can also offer things an established one may not, such as direct access to the product team; ask a reference whether that access was real.
Sources
See how Adira drafts in your voice and reads contracts from your side.
Explore the showroomWorking through a contract like this? Weave is Adira’s free tool to read, mark up, and connect any contract in your browser — no account needed.
Try Weave — free