What Customer Experience Can Tell Us
Naturalistic Evidence
Avery Martz | Updated August 2026
From the Testing & Methods Wiki
What Words Can Tell Us
Customer feedback has always been an important part of our product vetting process at Herbal Edge. We read the research and do a lot of work reviewing product formulations and testing products in the lab before they hit our shelves, but these things can't answer the most important questions people have before they buy something:
Did this actually help someone in real life?
Did it help them enough to justify the expense?
Customer feedback helps us bridge the gap between what might happen in theory and what actually happens in the real world. Customer experience is a form of naturalistic evidence. The limitations of this kind of evidence are clear. Correlation is not causation, rare side effects are very difficult to detect, and the placebo effect is powerful.
Naturalistic data generally can't separate those effects from the effects of the product itself. It can't necessarily tell us why something happened, but it can tell us if something is happening.
What Behavior Can Tell Us
Rather than relying solely on popularity to pick products, Herbal Edge uses customer feedback as a confirmatory test for products we already have good reason to think should work based on the fundamentals.
Customer feedback comes in two forms: what customers say and what customers do. If a product has good reviews but very few people return to buy it again, that behavior tells us something their words don't. The product may be effective but too expensive, difficult to use, or have some other problem that keeps people from using it in real life.
This is a very difficult test to pass. A product has to make sense on paper, survive our vetting and testing, and become popular among a meaningful group of customers who return for it repeatedly. That doesn't prove the product caused their results, but it gives us another kind of evidence that laboratory testing alone cannot provide.
What We Can't Tell You
We've always wanted to share this feedback with other customers, but customer reviews can create unusual problems in CBD. People don't naturally talk about their experiences in verbiage approved by the regulators at the FDA.
The FDA has repeatedly treated customer reviews on CBD websites as medical claims made by the company. When someone says our CBD helped with their anxiety, arthritis, migraines, neuropathy, insomnia, replaced a medication, or relieved symptoms they had dealt with for years, publishing that statement can make us responsible for the underlying medical claim.
The customer can make a sincere statement about their experience, but Herbal Edge can face regulatory consequences for publishing it. That means the feedback that best explains why someone values a product is often the feedback we're least able to share.
What We Publish
Therefore, our reviews come from real customers and reflect real feedback, but we frequently have to paraphrase them to remove or soften medical claims.
We preserve the customer's underlying experience without reproducing specific claims we're simply not permitted to publish. We don't invent experiences or make a customer's results sound better than what they reported. We generally have to make their claims less specific and less impressive.
What Reviews Are Hiding
It is common practice today for companies to feature only positive reviews, remove negative or even middling reviews, and then boast about near-perfect satisfaction or five-star review rates. I can tell you it is comically easy to manipulate reviews and alter what customers ultimately see, even on third-party platforms like Google, Amazon, and Yelp.
And there is a growing marketplace for paid testimonials on short-form platforms like Instagram. This is called "User Generated Content Advertising," and it is quickly becoming a major marketing medium for online brands. That is one reason people turn to long-form YouTube and Reddit for honest reviews nowadays.
Herbal Edge is about transparency. When we publish customer feedback on our website, we think you should understand the process behind it.
Hopefully someday we can find a way to share more of the feedback we've collected over the years unedited and allow people to speak more freely about their experiences.
Related Reading →
- Testing and Verification at Herbal Edge
- Hemp Is Legal. Talking About It Isn’t.
- "Data Driven" Science - A Crisis of Competence (Publishing Soon)
About the Author
Before launching Herbal Edge, Avery Martz operated one of Idaho's largest CBD retailers, reviewing products, working with independent makers, and helping hundreds of people learn about CBD.