Daniella Remy,1-2 MSc, ND , Adam Gratton,3 MSc, ND, and Kieran Cooley,3-5 ND (non-clinical)
ABSTRACT
Background: Healthcare professionals have the responsibility to educate their patients on natural health products (NHPs), yet the assessment of NHP quality throughout North America remains relatively subjective and prone to biases. This study aims to qualify multi-ingredient NHPs, based on the subjective and empirical attributes sought by naturopathic doctors (NDs) who regularly prescribe them.
Methods: This study was divided into two phases. Phase 1 involved virtual interviews with eight experienced NDs across North America. Phase 2 was an online survey of licensed and practicing NDs based on the key themes extracted from Phase 1.
Results: Using an inductive approach to qualitative analysis in Phase 1, four key themes were extracted: sourcing, labelling, monographs, and third-party testing, with each one having several sub-themes. Phase 2 revealed that sourcing was the most important theme, specifically from manufacturing companies that adhere to good manufacturing practices (GMPs), followed by products on which labels provide specific details of the active compounds. Third-party testing ranked third, especially if used to verify that ingredients match the label, and monographs should include referenced evidence on the therapeutic efficacy specific to the recommended dose of the product.
Conclusion: NDs believe that the strongest measure of complex NHP quality is the manufacturing company’s ability to adhere to GMPs. Third-party testing could be used to verify standards of quality with product details included in labels and ample referenced evidence in monographs.
Key Words Natural health products (NHPs), quality assessment, naturopathic medicine, good manufacturing practices (GMPs), third-party testing, North American NHP standards, naturopathic doctors (NDs) product preferences
The landscape of natural health products (NHPs) has significantly evolved, reflecting a growing integration of herbal and dietary supplements into healthcare regimes. These products contain vitamins, minerals, herbal and homeopathic medicine, and traditional medicines, aimed to support human health and wellness1,2. In an American survey conducted in 2017 and 2018, it was found that 57.6% of adults aged 20 and over had used an NHP within the past 30 days3.
Regulatory efforts in the United States, governed by the Dietary Supplement Health and Education Act (DSHEA) of 1994, underscore a commitment to safeguard consumer health while supporting industry innovation4. In the United States, dietary supplements are primarily regulated through post-market surveillance. Manufacturers are not required to demonstrate safety and efficacy to obtain product approval by the DSHEA unless they contain botanical ingredients.
Similar regulatory efforts in Canada were established by Health Canada under the Natural Health Products Regulations of 2004, setting a benchmark for the safety, efficacy, and quality of these products5. In contrast to the regulatory frameworks of the United States, Canada has established much stricter regulations for NHPs. In addition to obtaining product and site licensing requirements, adhering to good manufacturing practices (GMPs), adverse reaction reporting, clinical trial requirements, and strict labelling guidelines, Health Canada monitors both the manufacturing and post-marketing processes5.
Despite these efforts, challenges persist in ensuring the consistent quality and safety of NHPs across markets6. Recent incidents of poor-quality NHPs reaching the market have highlighted the limitations of current regulatory mechanisms and underscored the need for enhanced standards and methodologies for product evaluation7. The workshop on “Addressing Challenges in the Assessment of Botanical Dietary Supplement Safety,” hosted by the National Toxicology Program (NTP) in 2016,8 aimed to address these challenges by clarifying methods for establishing phytoequivalence, active constituent(s), and phytodynamics of botanicals, marking a significant step towards refining quality assessment practices.8 This resulted in the establishment of the Botanical Safety Consortium (BSC), which represents a collaborative effort to advance the science of botanical safety evaluation and to enhance the botanical dietary supplement industry’s ability to bring safe products to the market.9 This initiative is critical in bridging the gap between regulatory standards and the complex nature of botanical products, offering a framework for comprehensive safety assessment.
Nonetheless, there are incidents where complex NHPs have had hidden ingredients10,11,12 or where botanical supplements vary due to individual differences in the parts of the plant being used, where raw materials are grown, when they are harvested, or how components are extracted.13 As well described in a 5-part series,7 dietary supplement regulations have much room for improvement, especially for multi-ingredient and complex formulations.
Currently, individual assays in laboratory settings assess products for their identity, purity, and potency of ingredients, allowing a quality rating scale to compare products objectively. Unfortunately, such databases for multi-ingredient NHPs are lacking. The persistent issue of NHPs failing to meet quality and safety standards, despite regulatory oversight, necessitates a re-evaluation of current practices.14 This study, therefore, aims to contribute to this evolving landscape by developing an understanding of how NDs qualify multi-ingredient NHPs, in the hopes that the insights gained will help practitioners, industry, and policymakers hone in on the most important determining factors for NHP quality assessment.
This study employed a mixed-methods approach across two phases to develop a comprehensive understanding of factors influencing product quality from the perspective of NDs. The methods, modes of advertising, consent forms, and assurance of confidentiality were all approved by the Research Ethics Board (REB) of the Canadian College of Naturopathic Doctors (CCNM) in Toronto, Ontario.
NDs, especially with statutory registration/occupational licensing, are very likely to have extensive training in and to make frequent use of single-ingredient and multi-ingredient NHPs in their clinical practice,15 much more so than pharmacists,16 registered dieticians,17 and various conventional medical professionals.18 NDs were therefore selected as the main subject experts to provide insight on what factors must be considered to properly evaluate the quality of NHPs.
Participants were selected through targeted advertisements in naturopathic community newsletters and social media groups, ensuring that a diverse range of experiences and perspectives were represented. Participants were given CA$50 to thank them for their time.
A first come, first served basis was used to accept participants based on the inclusion criteria (must have a minimum of 5 years of clinical experience; no more than two participants from the same province or state as a previously interviewed participant; must have signed the consent form to participate in the study). Recruitment ceased once data saturation was obtained and any further interviews failed to produce added insights. Eight interviews were conducted.
Zoom calls were recorded, interviews were transcribed by a neutral third party and then submitted to the participant for accuracy. Each interview consisted of several pre-set questions, as well as questions that may have emerged from a previous interview. Such an inductive interview approach allows for flexibility in subsequent interviews on the themes and topics learned from previous interviews.
Transcriptions underwent a detailed thematic analysis after each interview was conducted. The data were meticulously coded, and emerging themes were identified and categorized. These emerging themes and topics were then used in subsequent interviews for further elaboration, until no new themes or topics surfaced. This rigorous process was informed by the principle of grounded theory, ensuring that the development of the quality assessment tool was firmly rooted in empirical data.
The themes and sub-themes that emerged from these interviews and the subsequent qualitative analysis formed the survey to be used in Phase 2. Preliminary drafts were then reviewed by interview participants to validate the findings and ensure they accurately reflected the NDs’ perspectives on NHP quality.
The survey phase targeted a broader pool of licensed, practicing NDs across Canada and the United States, with the aim of collecting at least 150 responses. Similar recruitment channels to Phase 1 were utilized, leveraging the study’s growing network and reputation within the naturopathic community to ensure a high response rate. Participants were entered in a draw for a prize of CA$250 to encourage their contribution.
Building on the qualitative insights from Phase 1, Survey Monkey (an online platform for survey data collection and analysis) was used to ask respondents to rate the importance of each identified theme on a 5-point Likert scale and rank the themes in order of importance. This quantitative approach allowed for the statistical validation of the qualitative findings, ensuring that the most significant factors influencing NHP quality were accurately captured. Forcing participants to rank the order of factors they use to assess NHP quality prevented all factors from being top-rated. The open-ended question “is there anything you feel is missing or worth considering?” was asked for each theme, allowing further insight with nuanced feedback that enriched the data set.
The survey responses were analyzed using statistical methods to identify mean scores and standard deviations for each theme. Open-ended responses were then analyzed qualitatively to support or enhance the quantitative findings.
Ethics approval was provided by the REB of CCNM in Toronto, Ontario.
Sixty-seven willing participants submitted an application to be interviewed, most of whom were rejected due to their geographic location being the same as a previous interviewee. Ultimately, eight virtual interviews were conducted with licensed NDs in Canada and the United States. Participants practiced in the following locations: two from Alberta and one each from British Columbia, Ontario, Nova Scotia, Vermont, Arizona, and Oregon.
By the eighth interview, data saturation was reached, indicating a comprehensive capture of perspectives on NHP quality. With each interview, themes clearly emerged, resulting in four main themes: Sourcing, Labelling, Monographs, and Third-party testing. Within each of these themes, several sub-themes were grouped to ultimately formulate the survey used in Phase 2 (see Appendix A for the full survey).
A total of 309 valid responses were obtained (after 13 were rejected because the respondent was not a licensed and practicing naturopathic doctor/physician in either Canada or the United States). Of these, 192 were Canadians from seven different provinces (see Table 1), and 117 were Americans from 18 different states (see Table 2).
TABLE 1 Canadian Participants and the Provinces in Which They Practice
TABLE 2 American Participants and the States in Which They Practice
As outlined in Figure 1, adherence to GMPs was considered important, with a notable 76 participants (31.2%) prioritizing it as the most significant factor, compared with 35 (14.4%) and 26 (10.7%) participants ranking it second or third. Additionally, the manufacturing process (e.g., cold-pressed, hydrolyzed) and preference for organic products were considered influential, with 39 (16.0%) and 40 (16.5%) participants deeming it the most noteworthy consideration, respectively.
FIGURE 1 Ranking of Items That Define Quality Sourcing
Figure 2 highlights the importance of identifying active constituents and/or standardized compounds in addition to the amount per capsule/serving, with a substantial 110 participants (47.0%) identifying it as the most crucial label information.
FIGURE 2 Ranking of Items That Define Quality Labelling
As summarized in Figure 3, the demonstration of therapeutic efficacy was deemed most important, with 57 participants (26.3%) rating it as a top priority. The effective dose detailed in the monographs followed closely, with 43 participants (19.8%) prioritizing it as the most significant factor.
FIGURE 3 Ranking of Items That Define Quality Monographs
Figure 4 summarizes the relative importance of evaluating third-party testing criteria for NHPs. The alignment of ingredients with those listed on the product label had 93 participants (44.5%) ranking it as the primary concern.
FIGURE 4 Ranking of Items That Define Quality Third-Party Testing
In the realm of dietary supplements, adherence to GMPs is vital for ensuring product safety and quality. GMPs, as set by regulatory authorities such as Health Canada and the US Food and Drug Administration (FDA), establish comprehensive requirements for manufacturing, testing, and quality assurance that help ensure dietary supplements are safe for consumption and free from contamination and inconsistencies. Despite GMPs being a top-ranked consideration by NDs in selecting high-quality NHPs, it is by far not the only consideration and, alone, would not be adequate in NHP quality assessment.
With the knowledge NDs have in botanical, nutrient, and orthomolecular compounds, it is not surprising that the largest number of surveyed participants would examine the active constituents and/or standardized compounds and their amount per capsule/serving when judging the quality of an NHP. These sub-themes stood above the botanical name, species, or part used described on a label, but all these sub-themes had some importance and are worth highlighting in high-quality NHPs. Practitioners must be highly familiar with each of these aspects, while industry and policymakers need to ensure transparency in these areas. This is emphasized if third-party testing can certify that ingredient labels were accurate in their reporting of included ingredients. The critical evaluation of third-party testing to verify product claims aligns with the increasing consumer demand for transparency and accountability in health product manufacturing.
The emphasis on therapeutic efficacy within product monographs is another finding that aligns with current trends in evidence-based practice. Practitioners prioritize products that not only comply with manufacturing standards but also demonstrate clear, substantiated and evidence-informed benefits. This is crucial in a market where the therapeutic claims of NHPs can vary widely, and where both traditional and scientific validation can significantly influence both clinical outcomes and patient trust. The study found that product monographs need to provide evidence for the product’s efficacy at the suggested dose. Thereafter, there is a relatively even distribution between the desire for monographs to discuss adverse effects and justify the ingredient form. This focus on efficacy echoes the broader healthcare industry’s shift towards rigorous clinical validation and the necessity for NHPs to meet these high standards to be considered viable therapeutic options.
Moreover, the prioritization of accurate labelling and the verification of ingredients through third-party testing cannot be overstated. In an industry plagued by instances of mislabelling and adulteration, third-party verification acts as a crucial safeguard, which was reflected as highly important in this study’s findings. Research has shown that products undergoing third-party testing are more likely to meet their label claims and be free from contaminants, thereby enhancing consumer trust and safety.19 This was consistent with the findings from this study where an even distribution was found between the desire to have third-party testing for product purity, unadulterated contents, ingredient purity, and the manufacturing process. Relying on third-party testing not only supports regulatory compliance but also aligns with consumer advocacy for greater transparency and accountability in dietary supplement production20 and is consistent with the professional expectations revealed in this study.
Though the United States has the world’s largest NHP market,21 the FDA’s Office of Dietary Supplement Programs continues to rely on manufacturers to ensure the safety and adequacy of their products.22 In Canada, on the other hand, NHPs are regulated more like drugs, with even more stringent legislation passed in June 2023.
With 192 Canadian NDs from 7 different provinces and 117 American NDs from 18 different states, the distribution of participants in this study ensured a wide representation from both countries, offering a robust foundation for analyzing their perspectives on the quality of NHPs.
The use of both interviews and surveys ensured a robust, evidence-based approach to tool development, enhancing its validity and utility for NDs assessing NHP quality. This methodology provided a detailed and justified approach to understanding and developing a quality assessment tool for complex NHPs. The mixed-methods design, combining inductive qualitative interviews with surveys, offered an understanding of the factors that influence NHP quality from the perspective of naturopathic practitioners. This comprehensive approach helps provide a unified framework for the assessment of complex NHPs.
The sample size in this study is small. However, the number of interviews was sufficient to draw necessary conclusions, so additional interviews would not likely offer a greater benefit. The study adopted an inductive approach to qualitative data collection, allowing themes to naturally emerge from the interviews. This method enabled a deep, contextual understanding of the factors NDs consider when assessing the quality of NHPs. The iterative nature of the interviews, akin to a Bayesian statistical method, but for qualitative research, enhanced the richness of the data collected. Interview participants were also given a week to review the draft version of the survey to ensure that everything they personally felt was crucial to include had, in fact, been included, thereby reducing any potential interviewer bias.
To provide adequate data for the statistical processing of Phase 2, Reise, Waller, and Comrey23 suggest that a sample size of 50 would be very poor, progressively increasing to a sample size of 1000 that they deem to be excellent. However, the sample size should reflect the number of variables being evaluated. Carpenter24 suggests the minimum standard used in communications research of 5 participants to every variable is sufficient. With the 4 main themes and 29 sub-themes in the survey, it can be argued that a total of 150 participants is sufficient.
Another bias could be underlined with the large number of survey participants from Ontario, reflecting the location where the study took place. The distribution of participants, though wide, does not adequately reflect the distribution of practicing NDs in Canada and the United States.
Future studies should consider the application of the Delphi technique, which involves multiple rounds of surveys to gather expert opinions and achieve consensus on the key criteria. By engaging a panel of experts in various industries (such as naturopathic medicine, pharmacology, and regulatory affairs), the Delphi technique could be used to systematically refine the priorities identified in this study, such as GMP adherence, detailed labelling, therapeutic efficacy, and third-party testing.
Though guidelines and policies can certainly be enhanced with these survey results, it is the compliance procedures that tend to be lengthy, costly, and challenging to execute for 100% of the market. We therefore suggest using consensus results from the Delphi technique to develop a digital quality assessment that ranks complex, multi-ingredient NHPs, much like some existing platforms offer for single-ingredient products. This system would provide a standardized framework for evaluating the safety, efficacy, and transparency of NHPs, facilitating informed decision-making for both practitioners and consumers. Future regulatory policies and practices for NHPs could also consider implementing a scoring system to enhance product quality assurances and product comparisons.
An ND’s selection of NHPs relies heavily on the manufacturing company’s reputation and its ability to adhere to GMPs and high-calibre extraction processes. The more transparency an NHP company can provide, the more likely the product will be considered to be of high quality. Sourcing, labelling, monographs, and third-party testing emerged as the four main themes to assess quality NHPs, with several sub-themes identified for each. For sourcing, participants placed the greatest importance on adherence to GMPs, followed by the inclusion of details of the manufacturing process (i.e., cold-pressed, hydrolyzed, etc.) provided on the product label or in the product monograph. Detailed labelling, including specific information on active constituents and/or standardized compounds, along with their amount per capsule/serving, was highly valued. The inclusion of monographs with referenced evidence on therapeutic efficacy relative to the recommended dosage was also deemed important. There was a strong preference for evidence-based information supporting product claims. Using external verification to ensure that product ingredients match their labels accurately establishes a sense of trust in NHP production. This underscores the value placed on product authenticity and purity. This analysis confirmed the relative importance of each factor in the quality assessment of NHPs, facilitating the development of a unified framework for assessing NHP quality that encapsulates these key elements.
1Electronic Multinational Research Group, Toronto, Canada; Shae.ai/ph360. me, Anthrophi Technologies, Wanchai, Hong Kong;
2Precision Health Group Pty Ltd, Albion, Queensland, Australia;
3Canadian College of Naturopathic Medicine, Toronto, ON, Canada;
4University of Toronto, Toronto, ON, Canada;
5National Center for Naturopathic Medicine, Southern Cross University, Lismore, NSW, Australia.
Special thanks to the participants in Phase 1 and Phase 2 of this project, as well as to Gelay Lagman for the detailed transcriptions.
We have read and understood the CAND Journal’s policy on conflicts of interest and declare that we have none.
This project received funding from the 2021 Student Innovation Fund, a competitive internal grant distributed by The Canadian College of Naturopathic Medicine. This fund was supported by Biotics Research and Fullscript.
1. Health Canada. Natural Health Products Regulations. Health Canada Food and Drugs Act SOR/2003-196. Accessed February 15, 2024. https://laws-lois.justice.gc.ca/eng/regulations/sor-2003-196/FullText.html
2. FDA. Dietary Supplements. U.S. Food and Drug Administration. Accessed February 15, 2024. https://www.fda.gov/food/dietary-supplements
3. Mishra S, Stierman B, Gahche JJ, Potischman N. Dietary supplement use among adults: United States, 2017–2018. NCHS Data Brief. 2021;399:1–8.
4. FDA. Dietary Supplement Health and Education Act of 1994. Accessed February 15, 2024. https://www.fda.gov/food/dietary-supplements
5. Health Canada. Natural health products. Quick facts. Accessed February 15, 2024. https://www.canada.ca/en/health-canada/services/drugs-health-products/natural-non-prescription.html
6. Thakkar S, Anklam E, Xu A, et al. Regulatory landscape of dietary supplements and herbal medicines from a global perspective. Regul Toxicol Pharmacol. 2020;114:104647. https://doi.org/10.1016/j.yrtph.2020.104647
Crossref PubMed
7. Brown AC. An overview of herb and dietary supplement efficacy, safety and government regulations in the United States with suggested improvements. Food Chem Toxicol. 2017;107(Pt A):449–471. https://doi.org/10.1016/j.fct.2017.07.050
Crossref
8. National Toxicology Program (NTP). Addressing challenges in the assessment of botanical dietary supplement safety (workshop). Accessed February 15, 2024. https://ntp.niehs.nih.gov/sites/default/files/ntp/about_ntp/bsc/2016/june/presentations/botanicals_508.pdf
9. Mitchell CA, Dever JT, Gafner S, et al. The Botanical Safety Consortium: a public–private partnership to enhance the botanical safety toolkit. Regul Toxicol Pharmacol. 2022;128:105090. https://doi.org/10.1016/j.yrtph.2021.105090
Crossref
10. Favretto D, Visentin S, Scrivano S, et al. Multiple incidence of the prescription diuretic hydrochlorothiazide in compounded nutritional supplements. Drug Test Anal. 2019;11(3):512–522.
Crossref
11. Liu Y, Lu F. Adulterated pharmaceutical chemicals in botanical dietary supplements: novel screening approaches. Rev Anal Chem. 2017;36(3):20160032.
Crossref
12. Rocha T, Amaral JS, Oliveira MBP. Adulteration of dietary supplements by the illegal addition of synthetic drugs: a review. Compr Rev Food Sci Food Saf. 2016;15(1):43–62.
Crossref PubMed
13. Shipkowski KA, Betz JM, Birnbaum LS, et al. Naturally complex: perspectives and challenges associated with Botanical Dietary Supplement Safety assessment. Food Chem Toxicol. 2018;118:963–971. https://doi.org/10.1016/j.fct.2018.04.007
Crossref PubMed PMC
14. Di Lorenzo C, Ceschi A, Kupferschmidt H, et al. Adverse effects of plant food supplements and botanical preparations: a systematic review with critical evaluation of causality. Br J Clin Pharmacol. 2016;81(4):673–685. https://doi.org/10.1111/bcp.12861
15. Steel A, Lloyd I, Foley H, Leach M. Prevalence and predictors of naturopathic practitioners’ self-reported practice behaviours: results of an international survey. Integr Med Res. 2022;11(4):100897.
Crossref PubMed PMC
16. Kwan D, Hirschkorn K, Boon H. US and Canadian pharmacists’ attitudes, knowledge, and professional practice behaviors toward dietary supplements: a systematic review. BMC Complement Altern Med. 2006;6(1):1–10.
Crossref
17. Dussault V, Marquis M. Natural health products: practices, perceptions and training needs of registered dietitians. J Food Res. 2017;6(1):87–94.
Crossref
18. Conway K, Ferstl C, Kumar M. Dietary supplement adulteration—knowledge, attitudes, and practices of California health care professionals: a cross-sectional survey study. Patient Educ Couns. 2023;114:107824.
Crossref
19. Bailey RL. Current regulatory guidelines and resources to support research of dietary supplements in the United States. Crit Rev Food Sci Nutr. 2020;60(2):298–309.
Crossref
20. Dwyer J, Saldanha L, Bailen R, et al. Commentary: an impossible dream? Integrating dietary supplement label databases: needs, challenges, next steps. J Food Compos Anal. 2021;102:103882.
Crossref
21. Chopra AS, Lordan R, Horbańczuk OK, et al. The current use and evolving landscape of nutraceuticals. Pharmacol Res. 2022;175:106001.
Crossref
22. FDA. Dietary Supplements Guidance Documents Regulatory Information. Accessed May 14, 2024. https://www.fda.gov/food/guidance-documents-regulatory-information-topic-food-and-dietary-supplements/dietary-supplements-guidance-documents-regulatory-information
23. Reise SP, Waller NG, Comrey AL. Factor analysis and scale revision. Psychol Assess. 2000;12(3):287.
Crossref PubMed
24. Carpenter S. Ten steps in scale development and reporting: a guide for researchers. Commun Methods Measures. 2018;12(1):25–44. https://doi.org/10.1080/19312458.2017.1396583
Crossref
Keep in mind that this is not a patient scenario but how you determine or rank the quality of one product versus another.
Please note that an inductive interview approach allows for flexibility in subsequent interviews. Themes and topics learned from one interview can influence future interview questions: E.g.: if a previous ND spoke of purity, a subsequent interview can ask “Other NDs have mentioned purity but that wasn’t in your top 5, why not?”
Correspondence to: Daniella Remy, 24 Fernbank Avenue, Toronto, ON M6H 1W2, Canada. E-mail: drremynd@gmail.com
To cite: Remy D, Gratton A, Cooley K. Establishing a unified framework for natural health product quality: insights from North American naturopathic practitioners. CAND Journal. 2024;31(4):3-10. https://doi.org/10.54434/candj.178
Received: 4 June 2024; Accepted: 18 September 2024; Published: 12 December 2024
Prior publication: Presented as a poster on Research Day 2022 at the Canadian College of Naturopathic Medicine.
© 2024 Canadian Association of Naturopathic Doctors. For permissions, please contact candj@cand.ca.
CAND Journal | Volume 31, No. 4, December 2024