arXiv · 2112.03945
Case Study: Evaluation of a meta-analysis of the association between soy protein and cardiovascular disease
Abstract
It is well-known that claims coming from observational studies most often fail to replicate. Experimental (randomized) trials, where conditions are under researcher control, have a high reputation and meta-analysis of experimental trials are considered the best possible evidence. Given the irreproducibility crisis, experiments lately are starting to be questioned. There is a need to know the reliability of claims coming from randomized trials. A case study is presented here independently examining a published meta-analysis of randomized trials claiming that soy protein intake improves cardiovascular health. Counting and p-value plotting techniques (standard p-value plot, p-value expectation plot, and volcano plot) are used. Counting (search space) analysis indicates that reported p-values from the meta-analysis could be biased low due to multiple testing and multiple modeling. Plotting techniques used to visualize the behavior of the data set used for meta-analysis suggest that statistics drawn from the base papers do not satisfy key assumptions of a random-effects meta-analysis. These assumptions include using unbiased statistics all drawn from the same population. Also, publication bias is unaddressed in the meta-analysis. The claim that soy protein intake should improve cardiovascular health is not supported by our analysis.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
S. Stanley Young, Warren B. Kindzierski, Douglas Hawkins, Paul Fogel, Terry Meyer. 2021-11-29. Case Study: Evaluation of a meta-analysis of the association between soy protein and cardiovascular disease. https://arxiv.org/abs/2112.03945
Cite the original work for its findings. Save a collection to share your selection of sources.