Path: EDN Asia >> Design Centre >> Consumer Electronics >> Scientific results evaluation: Systematic bias
Consumer Electronics Share print

Scientific results evaluation: Systematic bias

08 Mar 2016  | Ransom Stephens

Share this page with your friends

No experiment can be performed without some bias. Systematic uncertainties come primarily from unknown biases.

Comparison of systematic uncertainty and statistical uncertainty.

Estimating systematic uncertainties
Systematic uncertainties aren't sinister. When experimenters discover their biases, they find ways to remove them by using control tests such as double-blind testing. To estimate biases that they're unaware of, researchers (and readers of popular science should) try to estimate how the results change if the experimental techniques are altered. By approaching a measurement from different perspectives and comparing their best results to those that they know are biased, experimenters can determine the scale for the unavoidable inherent bias. From that scale, they can estimate their systematic uncertainty. For example, time domain and frequency domain measurements should always agree but never exactly, the difference measures the bias.

Here's another example, a much trickier one than is unlikely to be faced by engineers, physicists, or chemists. Consider the paper published in Science reporting that readers of literary fiction are more empathic than readers of genre fiction (like thrillers, mysteries, science fiction, romance, etc). If the researchers had examined their systematic uncertainty, they either wouldn't have made the claim or Science wouldn't have published it (Science has a reputation for publishing a few sensational results each year, probably to recruit subscribers).

They could have estimated their systematic uncertainty by performing the experiment with separate but consistent definitions of "literary" and "genre." They used excerpts of an anthology for literary fiction but could easily have used critically acclaimed but diverse works of fiction. By comparing their results under separate definitions of "literary," they could have estimated their systematic error and it probably would have dwarfed their statistical error.

The paper quotes results derived by commercial statistical analysis software that indicate a rather convincing level of statistical significance, Without the systematic uncertainty, however, their conclusions are as specious as my poll of the Raider Nation. There was, however, one claim they could have made: careful analysis of their results gives compelling if not quite conclusive evidence that reading fiction helps people develop empathy. Their experiment just didn't have the precision to resolve any dependence of the level of empathy on the category of fiction.

About the author
Ransom Stephens is a technologist, science writer, novelist, and Raiders fan.

 First Page Previous Page 1 • 2

Want to more of this to be delivered to you for FREE?

Subscribe to EDN Asia alerts and receive the latest design ideas and product news in your inbox.

Got to make sure you're not a robot. Please enter the code displayed on the right.

Time to activate your subscription - it's easy!

We have sent an activate request to your registerd e-email. Simply click on the link to activate your subscription.

We're doing this to protect your privacy and ensure you successfully receive your e-mail alerts.

Add New Comment
Visitor (To avoid code verification, simply login or register with us. It is fast and free!)
*Verify code:
Tech Impact

Regional Roundup
Control this smart glass with the blink of an eye
K-Glass 2 detects users' eye movements to point the cursor to recognise computer icons or objects in the Internet, and uses winks for commands. The researchers call this interface the "i-Mouse."

GlobalFoundries extends grants to Singapore students
ARM, Tencent Games team up to improve mobile gaming

News | Products | Design Features | Regional Roundup | Tech Impact