Environment researchers propose new guidelines for more transparency and reporting of research methods

Applied ecological disciplines experience a growing need for rapid decisions to be made under conditions of incomplete information. The use of expert knowledge as an alternative, or complement, to empirical data has been suggested as a possible approach to overcome this information shortfall. But the validity of this approach depends on the application of rigorous methods to minimize bias and ensure reliability. The ability to confirm a study’s methodological rigor depends on transparent reporting of the applied methods.

“A wide range of people, from researchers to practitioners and local resource users, have strong ecological knowledge,” Drescher says. “Much of this is not written up and is not in the form of empirical data. This knowledge is too important to ignore. But to use it properly, we have to apply solid methods, just as we would in empirical research”. This knowledge is needed especially in data-poor environments where resource use pressures and rapidly changing conditions require fast conservation actions, without having the time to collect large-scale or long-term empirical data sets.

To assess current levels of methodological transparency in expert knowledge use, Drescher and Edwards reviewed 296 published studies covering the period from 1996 to 2016. They used the field of forest ecology and management as a case study and as an example for ecological disciplines in general. They assessed each publication for reporting on 15 methodological criteria of elicitation and use of expert knowledge.

The results of their review show that over 50% of the published studies were reporting on just three of the 15 methodological criteria. Just 10% of studies reported on two thirds of the criteria.

Lack of methodological reporting is problematic because it means that it cannot be confirmed whether studies are reasonably unbiased and reliable. “There is a strong body of research on rigorous methods for the use of expert knowledge,” says Drescher. “Researchers have a great toolbox of methods to choose from. But how are we supposed to know what they are doing if they don’t tell us? How can we support researchers in choosing the best methods for their application if they don’t tell us what they are doing?”

To address this lack of methodological transparency, Drescher and Edwards are proposing guidelines for the reporting of methods in the elicitation and use of expert knowledge. The guidelines are phrased as a set of guiding questions, helping researchers to design their studies with methodological rigor and to decide which aspects of their methods should be reported. According to Drescher, “It is our hope that the research community finds these guidelines useful. We need to understand better what researchers are doing to improve current practice and make best use of the existing methods”.

Follow-up studies that examine how patterns of methodological applications and their reporting vary between different types of research and research fields would be useful. This would allow for the development of more tailored guidelines for these kinds of research applications.

Read the full article (freely available for a limited time):

Drescher, M, Edwards, E. A systematic review of transparency in the methods of expert knowledge use. J Appl. Ecol. 2018; 1–14. DOI: 10.1111/1365-2664.13275

Media contact:

Sam Toman, Faculty of Environment, University of Waterloo, Tel: +1 519 888 4567 x38100, Email: stoman@uwaterloo.ca