FDA Guidelines and Jadad Score

​

To ensure the robustness and accuracy of our relevance assessments and study design evaluations in the context of CAM-use pairs, we rely on the time-tested guidance of the FDA guidelines and the objectivity of the Jadad score. These pillars of evaluation empower us to contribute to a transparent and reliable assessment system, fostering scientific progress and safeguarding public health. By adhering to these guidelines and utilizing the Jadad score, we pave the way for a brighter future of evidence-based research and meaningful insights into the utilization of various CAM for diverse purposes.

​

FDA guidelines provide a comprehensive framework for conducting research and, for us, help us evaluate the quality of studies on how different ingredients and therapies can effectively aid with a given symptom or condition. Specifically, these guidelines help researchers structure their studies and methodologies, ensuring they are designed with adequate controls, sample sizes and endpoints. Moreover, the guidelines highlight the importance of considering potential biases and confounding factors that may influence study outcomes.

​

By closely following the FDA guidelines when evaluating a research paper, we establish a strong foundation for our assessments, where we try to enhance the trustworthiness of our findings.

​

We also integrate the Jadad Score into our evaluation system: the Jadad score is a widely recognized tool for assessing the methodological quality of clinical trials. This scoring system is particularly valuable when analyzing the efficacy of medical interventions, including CAM ingredients used for specific conditions.

​

The Jadad score evaluates studies based on three fundamental criteria:

​

• Randomization: This criterion assesses whether the allocation of participants into experimental and control groups is truly random, minimizing the risk of selection bias.

​

• Blinding: Blinding refers to the concealment of information about the treatment assignment from participants and/or researchers to prevent bias in the study results.

​

• Withdrawals and Dropouts: The Jadad score also considers the handling of participant withdrawals and dropouts to gauge the potential impact on the study's internal validity.

​

By incorporating Jadad criteria into our evaluation system, we can effectively and objectively assess the quality of study designs related to ingredient-use pairs. This score helps us differentiate between well-executed trials and those that may be susceptible to bias, confounding variables, or other methodological flaws.

​

          A closer look at our review design.

​

• Evidence of effectiveness:

  The abstract's conclusion is very positive about the effects of the ingredient on the condition: This point considers the overall sentiment of the abstract's conclusion regarding the ingredient's impact on the condition. More positive conclusions receive higher scores, while negative conclusions receive negative scores.

​

• Study design and overall quality:

  Was the study described as randomized? Was the study described as double-blind? Was there a description of withdrawals and dropouts? Was the study a meta-analysis or review? Is the study population representative?​ These criteria are drawn from a mixture of criteria in the Jadad Scale, and from FDA guidelines to assess the quality and design of the study mentioned in the abstract. Studies that are randomized, double-blind, with descriptions of withdrawals and dropouts, representative populations are generally considered more reliable and reproducible. Studies lacking these features are often less cited and less reliable.

​

• The paper is about humans:

  This point emphasizes the importance of human studies in determining the relevance of the ingredient for a specific condition. Human studies carry more weight as their results are directly applicable to human health.

​

• The abstract is about a minority:

  This criterion highlights the importance of studying diverse populations to ensure the ingredient's effectiveness for a broader range of people. Focusing on a very specific population may limit the generalizability of the findings.

​

The score is weighted according to the order of the criteria listed above, in descending order. Additionally, we take into account the number of citations as a measure of the impact factor of the study in its broader research context.

​

     What's next

​

Our system for retrieving and assessing scientific evidence makes use of Large Language Models (LLMs). These advanced models have proven extremely useful in providing insights when processing complex information such as our large dataset of scientific abstracts. Nevertheless, it remains imperative to acknowledge the limits that these models operate within.

​

In order to harness the full potential of language models while circumventing their limitations, we integrate them with a solid, transparent pipeline where we use them only after having pre-processed our database, and only to the extent we know they can provide useful information (e.g. answering Yes/No questions about a text).

​​