Back to class: Reflections from the 2023 CWTS Summer School
The Centre for Science and Technology Studies (CWTS) hosted their fifth Scientometrics Summer School (CS3) from September 18-26, 2023 in Leiden, the Netherlands. CS3 turns theoretical ideas into hands-on research by engaging scientometrics graduate students with data, methods, and techniques used in the field. ScholCommLab members Chantal Ripp, Maddie Hare, Leigh-Ann Butler, and Rémi Toupin attended this year’s event.
In this post, Rémi, Maddie, and Leigh-Ann reflect on their time at CS3, discussing what they learned about themselves, and how their experience will impact their current and future research.
Q1: How did CS3 connect to your studies and/or research?
Rémi Toupin: My work focuses on environmental information and public attention to environmental research. Coming from science and technology studies (STS) and science communication backgrounds, I am still new to the fields of information science and scientometrics—though I believe I have solid expertise in Altmetrics.
CS3 was an opportunity for me to fortify what I knew of scholarly communication and bibliometrics, as well as learn more about the best approaches to look at traces and indicators of research in various spheres. I aim to apply this knowledge to my future work to better look at the communication of environmental knowledge in news media, policy documents, and public contexts.
Maddie Hare: My doctoral research aims to reduce the misuse of scholarly metrics through the development of educational material. Over the duration of CS3, there were valuable discussions on different cultures of research evaluation and reform across the world which now inform how I think about the (mis)use of scholarly metrics in Canada.
I particularly benefited from learning about areas of connection and exchange between scientometrics and STS—the latter of which I previously had little exposure to. The instructors presented valuable perspectives, asking how we might foster agency and dialogue to shape science (e.g., values, legitimacy, and relevance). Learning about trends in scientometrics, future trajectories of scholarly publishing, open data sources and tools, and how to produce more responsible knowledge with quantitative methods has also influenced how I’m approaching my doctoral work.
Leigh-Ann Butler: As the Scholarly Communication Librarian at the University of Ottawa (UOttawa), it is imperative that I examine and research uOttawa scholarship and publication trends, so that the Library can make evidence-based decisions regarding its open access investments. The use of bibliometrics/scientometrics is crucial to this research.
CS3 provided me with the opportunity to critically examine scientometrics as a method, and explore more holistically how this approach can inform our understanding of the research system, from open scholarship broadly, to research assessment, publishing models, and science policy.
Through guest lectures and hands-on practical sessions, CS3 provided students exposure to innovative methods and the use of open data sources, like OpenAlex, as well as philosophical discussions about the history, present and future state of scholarly communication. I am already applying many of these lessons in my daily work, as I analyze uOttawa’s state of open access publishing and where we can head, thinking through the intersection of areas like research reform, and new publishing models.
Q2: How did you grow as a researcher during this time? What did you learn about yourself?
Maddie Hare: I attended CS3 to expand my data analysis skills. I was excited to learn about Google BigQuery (GBQ) and further develop my competency in querying using SQL dialect. During these sessions, I was delighted to notice a certain reflexiveness in myself. I was not so intimidated or discouraged by my skill level when working with data, but rather dove right into our GBQ exercises.
I felt that I had somewhat surmounted my previous anxiety towards programming and querying languages, allowing for curiosity and an openness to learning to emerge. Since the workshop, I have been applying GBQ and other tools in my work and hope to steadily improve my data skills over the course of my career.
Leigh-Ann Butler: I knew going into the school that one of my limitations is my expertise and ease in using quantitative methods. Although I’ve used bibliometrics in my Master’s thesis, and in various other projects, it often comes with hurdles. It is not easy for me! I enjoy discussing the philosophical aspects of this field, but the practical, hands-on applications are often tough. My goal was to learn new methods and test the boundaries of my comfort-level. Learning GBQ was challenging, but exciting. I realized quickly that writing a script does not come naturally for me, and it may never will!
What I did walk away with was understanding more practically how these tools can be used for different projects and that leveraging the expertise across a team is profoundly important to ensuring the success of a research project. I kind of know more about exactly where my place is on a team.
Q3: In what ways did you make meaningful connections?
Rémi Toupin: CS3 was an amazing opportunity to meet other researchers, whether it was in the teaching crew or the other students. I also had the chance to know some members of our lab a lot better, and even met a scholar whose work has been foundational to mine! Overall, I had a great time getting to know other researchers, their work, and their background. It showed just how interdisciplinary scientometrics is, and how it can branch out to so many interesting topics!
Leigh-Ann Butler: I second much of what Rémi says! I had an incredible time getting to know the other students in the class. We had a coffee break each day where we had the opportunity to get to know the other participants, and those teaching the course. This was incredibly valuable, to casually discuss our research, learn from one-another, and understand what other folks are doing. For example, there were a few other librarians there, and I had the chance to discuss with them their own approaches to scholarly communication, how they apply scientometrics currently, or how they wish to apply it. Also - our crew of Canadian participants really bonded during that time, and I loved the chance to get to know each other over that period of time.
To stay up-to-date on upcoming talks and events, subscribe to our newsletter.
From Ottawa to Leiden: ScholCommLab's Conference Highlights
Over the last few months, ScholCommLab members took part in over six conferences, including the: International Conference on Science, Technology, and Innovation Indicators (STI 2023), International Communication Association Conference (ICA 2023), and Bibliometrics and Research Impact Community Conference (BRIC 2023).
In this blog post, we’re sharing highlights from each event.
International Conference on Science, Technology, and Innovation Indicators
Nine ScholCommLab members—Isabelle Dorsch, Maddie Hare, Marc-André Simard, Rémi Toupin, Kathleen Gregory, Stefanie Haustein, Chantal Ripp, Leigh-Ann Butler, and Isabella Peters—participated in the 27th International Conference on Science, Technology, and Innovation Indicators (STI 2023) in Leiden, the Netherlands from September 26-29, 2023.
A special session on Metrics Literacies was organized by Stefanie Haustein, Heather Woods, Maddie Hare, Isabelle Dorsch, and Carey Ming-Li Chen. This incubator session invited community partners in bibliometrics—including researchers and analysts, data providers, and librarians—to take ownership of metrics education and generate ideas on how to effectively and efficiently communicate metrics to the wider audience of scholarly users.
Prior to the conference, ScholCommLabbers Chantal Ripp, Maddie Hare, Leigh-Ann Butler and Rémi Toupin attended the 8-day Centre for Science and Technology Studies (CWTS) Scientometrics Summer School. “The CWTS Summer School was an excellent opportunity to review critical skills in bibliometrics, including visualizing collaboration patterns in social networks,” says doctoral candidate Chantal Ripp.
“The course also helped me to reflect critically on my key research questions and how we ultimately operationalize concepts in scientometric studies," she says. "It was also a great opportunity to learn from many of the experts in our field. Being able to converse and ask them questions directly is something I really appreciated from the in person experience.”
Bibliometrics and Research Impact Conference
Taking place from June 7-8, in Ottawa, Canada, ScholCommLabbers Juan Pablo Alperin, Leigh-Ann Butler, Chantal Ripp, and Maddie Hare attended BRIC 2023. Invited librarians, information specialists, and scholarly communication researchers from Canadian academic institutions discussed work in the areas of bibliometrics and research impact assessment.
Juan’s keynote focused on the global and diverse world of research and scholarship, and Leigh-Ann’s talk covered her Master's research on the amount of article processing charges academic authors paid to the five for-profit publishing houses between 2015 and 2018 (now published in Quantitative Science Studies). “It was so exciting to connect with such a vibrant, diverse community in person,” says Leigh-Ann.
International Communication Association Conference
Taking place from May 25-29, in Toronto, Canada, ICA 2023 brought together communication scholars from around the world. This year’s theme, “Reclaiming Authenticity in Communication,” focused on the relational, social, political, and cultural implications of authenticity in public discourse and popular culture across the globe.
ScholCommLab's Alice Fleerackers (recent PhD graduate) discussed in-progress research that examines what audiences think “preprint” means when they see it in the news and why it matters for science journalism and science communication. It builds on her previous research with Chelsea Ratcliff, Rebekah Wicke, Blue Harvill, Andy King, and Jakob Jensen, which found that only one in four people are able to define this term accurately. During the presentation, Alice learned that it isn’t just news audiences who are unsure what preprints are. Many of the scholars in the audience were also unfamiliar with the term!
Alongside Raul Ferrer-Conill (Associate Professor at University of Stavanger), Alice also organized a mentorship program for ICA’s Journalism Studies Division designed to connect early career and senior scholars. Now in its second year, the program provides opportunities for mentees to find support and guidance in career development and develop informal relationships with more established researchers.
“Running the mentorship program has been one of the most rewarding parts of attending the ICA conferences,” she says. “It’s always inspiring to see so many scholars volunteer their time to help others succeed.” In addition, Alice enjoyed nerding out about journalism and science communication, and reconnecting with some familiar faces from around the globe.
To stay up-to-date on upcoming talks and events, subscribe to our newsletter.
Three questions with... Maddie!
Our lab is growing! In our Three Questions series, we’re profiling each of our members and the amazing work they’re doing.
Today's post features Maddie Hare, a doctoral student in the Digital Transformation and Innovation (DTI) Program at the University of Ottawa—and the newest member of the ScholCommLab. In this interview, she tells us about her work on improving the understanding and use of scholarly metrics across academia, how interacting with peers advances our knowledge, and her secret to the most delicious chocolate cake.
Q#1 What are you working on at the lab?
I am currently working on the SSHRC-funded Metrics Literacies project. The project aims to reduce the misuse of bibliometrics among researchers and research managers through the development, testing, and distribution of open educational resources (OERs). The first step towards achieving this goal is an organized attempt at educating the broader academic community. Our interdisciplinary team of bibliometricians, science communicators, media producers, and education technology scholars are conducting a randomized control trial, to test the efficacy of different video formats (e.g., talking head, animation) for teaching wider audiences about scholarly metrics. The h-index is being used as a case study for the video content. By fostering Metrics Literacies, this project seeks to empower researchers and institutions to critically and ethically engage with indicators.
Our Metrics Literacies team will be running a Special Session at the 27th International Conference on Science, Technology and Innovation Indicators (STI 2023) in Leiden, the Netherlands on September 28 at 16:00 CET. The objective of the session is to empower the bibliometrics community to take ownership of metrics education. The hands-on session will work as an incubator of ideas on how to communicate the knowledge of bibliometrics experts effectively and efficiently to the wider audience of users of scholarly metrics and facilitate collaborations between various stakeholders, including bibliometrics researchers and analysts, data providers, and librarians.
Q#2 Tell us about a recent paper, presentation, or project you’re proud of.
In Spring 2023, I had the opportunity to sit on a panel for the third annual Dalhousie Science Society (DSS) Science Symposium and participate in a mentorship session after the panel. I thought it would be useful to expose undergraduate science students to our interdisciplinary field of Library and Information Studies. It was so delightful to chat with students who had never heard or considered librarianship or information science as a profession, and were intrigued about its intersections with science and technology. We chatted about technological developments like artificial intelligence, science communication, and everything to do with grad school! While librarians are not always expected to be experts in their liaison field, having a multitude of perspectives and educational backgrounds in our profession is important. It was really gratifying to engage with students early on in their careers.
In addition to helping run the Metrics Literacies Special Session at STI 2023, I will be presenting the preprint "Do You Cite What You Tweet? Investigating the relationship between tweeting and citing research articles" (available in arXiv). This work conceptualizes Altmetrics as indicators of information behaviour that are better understood when considering the relational characteristics of the actor(s), the work(s) acted upon, and the relationships and interactions between them that exist outside of the context in which the act occurred. I am looking forward to engaging in fruitful discussions about this study with the scientometrics community at STI 2023.
Q#3 What’s the best (or worst) piece of advice you’ve ever received?
When I started the Master of Information program at Dalhousie University, an Information Management professional sagely advised me that although academic excellence should be prioritized, relationships are the lifeblood of our field. With the hindsight of having completed my degree, I truly agree that it is the people around you that stimulate innovative ideas through collaboration, galvanize change through collective action, and support and root for you through every step of what will be a journey unique to you. I have learned so much from my colleagues, peers, and students, and am grateful that I received affirmation early on that relationships are keystones in our efforts to advance knowledge. I would pass on such counsel to anyone who is starting on an academic path: seek and nurture community!
Also, that the secret to the most delicious and moist chocolate cake is boiling water and a bit of instant espresso. Magnifico!
Find Maddie on Twitter at @madelainehare or check out her publications on ORCID.
Fatou Bah on the Bromley Memorial Event
Fatou Bah, master's student in the ScholCommLab, attended the 2023 Bromley Memorial event in Washington, D.C. alongside invited graduate students from the University of Ottawa (UOttawa).
Organized by the UOttawa Institute for Science, Society & Policy (ISSP) and the George Washington University Institute for International Science & Technology Policy (IISTP), the Bromley Memorial event provides an opportunity for graduate students to discuss and exchange ideas with senior science and technology policy advisors from various countries, sectors, and levels of government.
In this post, Fatou reflects on how the event impacted her personal and professional development.
Attending the Bromley Memorial Event with students from UOttawa and George Washington University was an incredibly enriching experience. We got the opportunity to meet and exchange ideas with science and technology policy advisors from NASA, the American Association for the Advancement of Science (AAAS), and the Canadian Embassy in Washington D.C.
Our visit to the Canadian Embassy was the most impactful part of the trip. We got the chance to visit the beautiful Embassy building, filled with art from various Canadian artists, showcasing the diversity in our country and highlighting Indigenous art and artists.
Through our meeting with senior Embassy officials, I learned about the field of science diplomacy which combines both my interests in science policy and diplomacy. I see myself pursuing a career in science diplomacy, more specifically in open science policy, to provide equitable access to science to all. We were also briefed on an upcoming treaty that would facilitate collaboration between researchers in the U.S and Canada. Through this treaty, I hope to collaborate with fellow researchers in the fields of open science and critical disability studies.
The keynote by Dr. Bhavya Lal (Associate Administrator for Technology, Policy and Strategy at NASA) provided fascinating insight into NASA’s Artemis campaign. I gained knowledge of the policy implications for large projects requiring collaboration from several countries, such as the International Space Station, and collaboration with private industries, such as the creation of human landing systems. I look forward to learning more about the research being done on human space travel, and how it may intersect with my own research interests.
I hope to participate in future Bromley lectures, to continue exchanging ideas with like-minded students and science policy advisors.
This post was featured in the ISSP blog by Geneviève Dunn.
Subscribe to our newsletter to receive updates on lab events and research!
New preprint explores tracing data reuse and citations
In our digital era, scientists are certainly sharing and reusing open data. Yet it remains unclear how widespread data reuse and citation practices are within academic disciplines, and why scientists cite—or do not cite—data in their research work.
In a recent preprint from the Meaningful Data Counts project, Kathleen Gregory (postdoctoral researcher at the University of Vienna and University of Ottawa) and fellow ScholCommLab members—Anton Boudreau Ninkov, Chantal Ripp, Emma Roblin, Isabella Peters, and Stefanie Haustein—surveyed nearly 2,500 academic authors to explore their practices, preferences, and motivations for reusing and citing data, and how these practices vary by discipline.
In this interview, we ask Kathleen about how she got involved in study, why some researchers cite and reuse data while others do not, and how her work informs data citation policies and standards in the scholarly community.
Q1. What drew you into studying data citations, sharing, and reuse?
This project combines two of my long standing interests: research data practices and bibliometrics. I’ve been interested in data sharing since my early days as an academic librarian when open access to data from federally funded research was first being discussed. This interest eventually led me to do a PhD exploring how researchers discover, make sense of and reuse data. Before doing my PhD, I spent some time focusing on bibliometric services in libraries and thinking about the responsible use of metrics in academic assessments. Exploring data citation practices, and their repercussions, provided a nice opportunity to merge these interests.
Q2. What questions or challenges were you setting out to address when you started this work?
One of our big questions in the project as a whole is to learn more about how researchers cite data in their work and for which purposes, and to think about how this varies by academic discipline. We started out by looking at this question bibliometrically in work led by Anton with DataCite. The survey which we report on in the preprint gave us a chance to look at this question from a different angle and to ask researchers not only about how they cite data (spoiler - it isn’t always with a citation in a reference list) and what they actually cite (another spoiler - it isn’t always a dataset) but also to learn more about why they do so.
One of the strengths of the survey is the care that we took with constructing a sample which, to the best of our efforts, is representative of academic authors by discipline, as represented in the Web of Science. This took many discussions to think about how best to do this, and much planning, but it also allows our findings to be stronger and perhaps more generalizable to the broader population.
Q3. What motivates researchers who cite data to do so?
Across the entire sample, we found that the majority of reasons might be construed as motivations which reflect ‘ideal’ research practice (e.g. to show intellectual debt), to help others to find data, or to support the validity of their research claims. Few respondents indicated that external factors, namely being advised to do so by journals or publishers, was a motivating factor in their decision to cite data.
We did find some significant disciplinary differences to this question, though. For example, social science and humanities (SHH) researchers selected that they cite data to acknowledge intellectual debt more frequently than expected. One possible explanation for this could be tied to common purposes for which our SSH respondents reuse data (e.g. to serve as the basis for a new study or to integrate sources to build an argument).
Q4. How do researchers prefer others to cite their work?
Around 84% of all respondents indicated that they would like others to refer to a related article or publication, followed by the data source (55%), and then the data themselves (46%). I found it super interesting, though, that nearly three-quarters of all respondents selected more than one option for this question. For example, about 950 respondents selected wanting other people to cite or mention both a related research article as well as the data themselves. This complicates the existing narratives suggesting that data citations alone may incentivise sharing data and suggests that a combination of citing different ‘data objects’ may be preferred by researchers.
Q5. In the survey, more than 450 respondents said that they do not reuse data. Why do some researchers not reuse data?
We asked people to self-identify as being a person who reuses data or one who does not. This classification itself is perhaps a bit problematic, as we know from past work in Science and Technology Studies that ‘usage’ of something—be that a car or the Internet or research data—exists on a spectrum. A researcher may reuse data to teach a class once per year, or they may be involved in a years-long comparative study and be reusing data daily, or they may not reuse data at all.
That said, the most frequently selected options were that reusing data was not relevant to their research methods or in their research communities. Other less frequently selected reasons—such as a lack of relevant data or difficulties finding data—indicate that perhaps some people may have tried to reuse data previously. I bring this up as I think it is important to remember that this variation in practice exists, that not everyone is a re-user of data, and that other ways of conducting research are also valuable.
Q6. What are your most interesting or surprising findings?
Perhaps most surprising to me at a high level was how much our data had to say about the practices of SSH researchers. We found that SSH researchers are the only disciplinary groups who prefer that others cite or mention their own data, as opposed to other data objects (e.g. an article or a collection of data). This contrasts past scientometric work showing that SSH scholars cite ‘data studies,’ rather than individual data files.
We were also able to identify unique differences between social scientists and humanities scholars, such as the tendency for humanities scholars to integrate data more than most other disciplinary groups or the pattern of social scientists to refer to related publications (rather than data) less than other groups.
Q7. Why is your study important?
Our study brings up what I think is a key question when designing policies and standards in the research data space in general. Our data show that data citation is rooted in longstanding disciplinary practices (e.g. the use of footnotes in the humanities), choices about standards (e.g. the use of APA citation guidelines in the social sciences) and choosing what to cite (e.g. the common practice of citing publications in addition to or instead of data).
Our data also show that most researchers who are citing data (in any way) report doing so for reasons in line with responsibly conducting research. This suggests that their current citation practices may be meeting both the norms of their communities and their own ethical principles. If this is the case, the questions I see which we need to consider are: When (and how) do we meet researchers where they are? When should their research practice be adapted to current technical requirements and our own recommendations for data citation, and when should those requirements and recommendations be adapted to reflect researchers’ actual practice?
It’s an important reminder that we need to take our time and involve research communities in the development of policies and systems.
To stay up to date with the Meaningful Data Counts project, check out Zenodo for all research outputs and sign up for our newsletter.
Behind the scenes: New research on doctors who vlog
Doctors are moving out of their clinics and taking over the digital world. Many physicians are engaging in digital media, sharing their expertise and opinions on platforms such as Instagram, TikTok and YouTube. But little is known on how doctors are mediating health research on social media.
Noha Atef, a postdoctoral fellow at the ScholCommLab, led a series of qualitative studies on doctors who create video blogs (or vlogs) on YouTube. Alongside lab members Alice Fleerackers and Juan Pablo Alperin, they explored why Egyptian doctors use vlogs, how they present themselves online, and what makes health vlogging rewarding to them.
In this interview, we ask Noha about how she got involved in the research, unexpected findings from her work, and how her studies help inform the field of health communication.
Q1. What got you interested in studying how doctors vlog?
During the pandemic, many people recorded and shared vlogs of their health care routines—from skincare to diets—on social media. I noticed these vlogs received thousands of views, even if the health advice was misinformed or lacked quality evidence. This aroused my curiosity to verify the content of viral health vlogs. I started following physician vloggers, including dermatologists, dietitians, and pharmacists, who I perceived to be more reliable sources of health information when compared to civilians.
When I started my research fellowship at the ScholCommLab, there was ongoing research about journalism and health communication. I wanted to expand on the work by focusing on “citizen journalists,” particularly Egyptian doctors on social media who discuss health information. My studies on doctors who vlog explored the people behind the videos—why physicians use (and what they gain from) vlogging, how they present themselves online, and how they discuss and present research in these videos.
Q2. Why do doctors make YouTube videos?
While conducting the interviews, participants told me about their busy schedules and how their time translates into money. It also made me wonder why they would allocate time to make their vlogs, growing their YouTube channels and social media accounts.
“Uses and Gratification Theory” was helpful to answer these questions and explore the benefits that physicians get from health vlogging. In this preprint study, we learned that participants created vlogs to achieve self-focused goals (e.g., answering patients’ recurring questions to save time in the clinic, promoting themselves) as well as society-focused goals (e.g., spreading medical awareness, addressing cultural and social issues related to health and wellbeing).
Q3. How did the doctors present themselves on their vlogs?
We used “Goffman’s Theory of Presentation of Self” to understand what face doctors wear on their blogs. Goffman proposes that humans are actors who wear “faces” and act differently according to what would keep you accepted in the community you present in. For example, the “face” we wear at work may not be the same we wear with friends or at home.
In this preprint study (forthcoming in the International Journal of Communication), we found that our participants embodied the characteristics of two types of people: health professionals and influencers (as they were popular vloggers, some of them with millions of followers). We found that participants wore multiple faces during their vlogs, emphasizing four traits: approachable, knowledgeable, pedagogical, and popular. Their self-presentation appeared to be a negotiation between two roles: part influencer, or social media content creator, and part doctor, or health service provider.
Q4. What was your most surprising finding?
My most surprising, and important, finding (now available in a preprint study, currently under review) was that physician vloggers scarcely referred to medical research in their videos. We found that these vloggers only cited academic research in the videos about medical controversies, new treatments, misinformation, or common medical mistakes.
Many of the doctors felt that talking “scientifically” could be detrimental to their audience, if the people cannot decode the science. They believed their audiences would not understand the evidence due to their low educational status, weak research skills, and language barriers (as most medical research is published in English rather than Arabic).
These findings highlight the challenges of communicating science in societies with a low level of education. In these settings, science communicators—including physician vloggers—should consider focusing on simplifying the research, translating and connecting evidence-based information to the cultural context, rather than solely relying on citations.
Q5. What did you gain from using mixed method approaches to address your research questions?
Prior to conducting the focus groups, we asked 200+ Arab social media users about how they select the sources of their health information online. We used this information to generate graphs and shared them with the health vloggers to start a discussion about their audience. This led to insightful discussions with participants, allowing them to go deeper and share their observations on how their audience interacts with their content. I also believe in the power of interviews. I can create depth in them, especially with sharing the language and culture of the participants (as I am Egyptian, too).
Q6. What kind of response have you received on your findings from doctors?
The twelve doctors who participated in the study really enjoyed the findings from each of the papers. One of the participants is now in academic research and particularly liked how the research was conducted with Egyptian health professional vloggers. According to the participant, there were many discussions in their research institution about the use of social media for medical awareness. My work may provide some insight into this phenomenon, from an Egyptian context.
Q7. Why is your research important?
My research is one of the first projects to investigate Arab health vloggers. It adds a qualitative perspective to a quantitative-dominated research area, identifying the active role participants played as both sources of data and critical reviewers of the preliminary results. The work also fills knowledge gaps in the science communication with respect to health influencers and their self-presentation, the benefits of vlogging to the vloggers and, maybe most importantly, the relevance of citing scientific evidence.
Q8. What’s next for you or your work? What are some possible real world applications?
I hope that further studies will be carried out on the ethics of vlogging health and how they are influenced by cultural and legal contexts. I’d also like to see more qualitative research and ethnography studies on health vlogging. I’m very much interested in mixed-method research that tracks the virality of misinformation, which may inform how and when health vlogs get their most views.
Sign up for our newsletter to receive updates on the health and science communication projects and other lab research!
How is emerging health research communicated online?
In the slow, unpredictable world of journal publication, preprints—unreviewed published papers—offer a mechanism for rapidly communicating health research with the scholarly community.
Historically, media coverage of preprints was discouraged in journalism, due to potential concerns about reporting flawed, biased, or provisional research. In recent years, the urgency to address the pandemic and absence of relevant peer-reviewed studies led to a surge of media coverage of COVID-19 preprints.
In 2020, the ScholCommLab initiated a SSHRC-funded project Sharing Health Research. It aims to explore how, where, and among whom health information circulates online, in the form of research publications, preprints, news stories, social media posts, and more.
Alice Fleerackers—PhD candidate and researcher in the ScholCommLab—has since led a series of qualitative and quantitative studies to understand how preprints are portrayed, understood, and communicated online in the pandemic era.
Read more to learn about Alice’s work: how preprints are framed by media outlets, what journalists’ have to say about preprints, and how the public reacts to emerging health research.
Framing preprints as uncertain in the media
In a study (published in Health Communication in 2021), Alice and fellow ScholCommLab members—Michelle Riedlinger, Laura Moorhead, Rukhsana Ahmed, and Juan Pablo Alperin—examined more than 450 stories mentioning COVID-19-related preprints to understand how media outlets reported on these unreviewed studies.
Some outlets—such as Medscape and Wired—almost always framed the preprints they mentioned as uncertain, describing them as unreviewed, preliminary, or in need of further verification or explicitly labeling them as “preprints.” Other outlets—including The Conversation and New York Times—emphasized uncertainty less than half of the time.
As findings may change from the preprint to the final peer-reviewed journal, transparency in communicating scientific uncertainty becomes crucial to avoid confusion within and beyond the scholarly community.
But to Alice and her coauthors, it still wasn’t clear what might be going on behind the scenes. What practices do journalists’ use to communicate about preprints?
A careful calculation
In a follow-up study, published in PLOS ONE in 2022, Alice and her co-authors—Laura Moorhead, Lauren Maggio, Kaylee Fagan, and Juan Pablo Alperin—explored journalists’ use and perception of preprints. Kaylee—a former journalist—led interviews with 19 journalists, asking questions related to the benefits and risks of covering preprints; how journalists’ find, verify, and communicate preprints; and how the pandemic affected their use.
The researchers found that deciding to cover preprints is a careful calculation, where journalists weigh the potential public benefits and accessibility of preprints against the risks of spreading misinformation.
Journalists also described maintaining “extra skepticism” when covering preprint studies, relying on strategies such as reaching out to scientists unaffiliated with the research to ask for a critique of the work. Even with these strategies, many journalists also felt they lacked the expertise or time required to fully vet the research.
Implications for the public
With increased coverage of preprint research in the media, Alice and her colleagues—Chelsea Ratcliff, Rebekah Wicke, Blue Harvill, Andy King, and Jakob Jensen—were curious to learn more on how the public interpreted unpublished or unverified information.
In a mixed methods study (published in Health Communication in 2023) led by Chelsea, they asked US adults to read a news story on preprint COVID-19 vaccine research. Some participants read a version of the story in which scientific claims were “hedged” or qualified with terms like “may,” “might,” or “suggest” to convey their tentativeness, while others read an unhedged version. In addition, half of the stories explicitly mentioned the preprint status of the research (along with a brief definition), while the other half simply referred to the research as a “study.”
Those who read the hedged articles reported less favourable attitudes towards the vaccine and found the scientists and news reporting less trustworthy overall. In contrast, describing the study as a preprint did not influence participants’ attitudes or beliefs in any significant way, perhaps because they did not understand what the term meant. A qualitative analysis of the survey data revealed that only about 1 in 4 participants was able to define the term ‘preprint’ accurately.
Journalists are often encouraged to include these disclosures of preprint status in their stories to prevent audiences from putting too much trust in findings that could change in the future or fail to pass peer review. But the study suggests that more needs to be done to help audiences fully grasp what the term ‘preprint’ means.
Communicating about preprints online
Ensuring news stories provide the context needed to support audiences in evaluating preprint research is especially important in a digital media environment. An additional study found that citizens often shared COVID-19 preprint stories covered by The Conversation—an outlet that seldom mentioned the unreviewed or preliminary nature of these studies—on social media.
That is, the public—with limited knowledge of what preprints are—shared links to stories, unaware they were amplifying preprint research. Circulating preprints may be helpful at times (e.g., providing life-saving information at a time of need). But in some cases, controversial and low quality preprints make the news, which may harm citizens—especially if the unvetted nature of the research was not transparently communicated.
"It's up to all of us to develop responsible and ethical norms around how, when, and why to use preprints."
- Alice Fleerackers
Communicating preprints beyond the pandemic
According to Alice, “It’s up to all of us to develop responsible and ethical norms around how, when, and why to use preprints.” Both journalists and scientists agree that preprints are here to stay. But these studies suggest that we need better strategies to support journalists in using them responsibly.
Alice is expanding on her findings on journalism and preprints as part of a new project—the Value of Openness, Inclusion, Communication, and Engagement for Science in a Post-Pandemic World (VOICES). This international, interdisciplinary team will explore how journalists use other Open Science outputs, such as data sets and Open Access publications.
The team hopes to offer insight on how the scholarly and journalism communities can better work together to provide trusted public health information and support public engagement with science and evidence, even when misinformation and scientific uncertainty are high.
For updates about the VOICES project and other lab research, sign up for our newsletter.
Three questions with... Marc-André!
Our lab is growing! In our Three Questions series, we’re profiling each of our members and the amazing work they’re doing.
This week's post highlights Marc-André Simard, a doctoral student in Information Science at the University of Montreal and member of the ScholCommLab. In this post, he shares his thoughts on work in the lab, the global use of open access literature, and stepping out of his comfort zones.
Q#1 What are you working on at the lab?
I am involved in the Open Access and Open Science project with Stefanie Haustein and other members of the team. We are almost done with our study led by Leigh-Ann Butler on the oligopoly of scholarly publishers and how they have been leveraging article processing charges (APCs) to squeeze even more money out of the pockets of the research community. We have recently shared a preprint version of our article on Zenodo which has made quite a splash on social media such as Twitter. The final version of this paper should be available in the next few months.
I am also working on my PhD thesis under the supervision of Vincent Larivière (University of Montreal) and Stefanie Haustein. I am studying compliance to open science policies such as open access and open data policies.
Q#2 Tell us about a recent paper, presentation, or project you’re proud of.
After publishing an article based on my master’s thesis on national differences in the use of open access literature, I was approached by PLOS to discuss some of the implications of my findings and the importance of open access for low-income countries through their blog. Our results showed that low-income and lower-middle-income countries, especially in Sub-Saharan Africa, tend to publish and use open access literature more than the rest of the world.
I really enjoyed the experience of transforming the fruits of two years of labour into a very short text aimed at a global audience, it really took me out of my comfort zone! It also made me realize the importance of science communication: what good is science if it cannot be communicated in a way that is clearly understandable by the rest of the population and by the people who make the decisions?
Q#3 What’s the best (or worst) piece of advice you’ve ever received?
I love this quote from The Brothers Karamazov by Fyodor Dostoevsky:
“Besides, nowadays, almost all capable people are terribly afraid of being ridiculous and are miserable because of it.”
All my life I have been a very serious person (even when I was a child) that was afraid of finding myself in embarrassing situations. The older I get the more I realize that unless you do something that is almost cartoonishly unhinged, most people will enjoy your company more if you are not afraid to step out of your comfort zone. Sing the karaoke song at two in the morning, I promise you will not regret it!
Find Marc-André on Twitter at @M_Simard_ or check out his publications on Google Scholar or ORCID.
(Re)connecting in Granada: Attending the 2022 International Conference on Science, Technology and Innovation Indicators
This September, the 26th International Conference on Science, Technology and Innovation Indicators (STI 2022) was held in Granada, Spain and the ScholCommLab left its mark at the first in-person edition of the conference since the COVID-19 pandemic.
Seven lab members—Anton Ninkov, Chantal Ripp, Isabella Peters, Kathleen Gregory, Leigh-Ann Butler, Marc-André Simard, and Stefanie Haustein—attended, presented and chaired sessions at the conference. For some of them, this was the first time they met in person. “I was especially excited to meet Kathleen, Anton, Chantal and Leigh-Ann,” says Isabella Peters, professor at CAU Kiel University. “We have been working together on the Meaningful Data Counts projects for almost two years and haven’t met personally until the conference. It was a blast to finally meet the people you spend so much time with on Zoom.”
-
Anton Ninkov presenting on "Mapping Metadata - Improving Dataset Discipline Classification" at the 2022 STI conference. -
Marc-André Simard presenting on "Geographic differences in the uptake of diamond open access and APCs" at the 2022 STI conference. -
Leigh-Ann Butler presenting on "The Oligopoly's Shift to Open Access Publishing: How For-Profit Publishers Benefit from Gold and Hybrid Article Processing Charges" at the 2022 STI conference. -
Kathleen Gregory presenting on "Surveying practices of data citation and reuse across disciplines" at the 2022 STI conference.
The conference, first held in 1988 at Leiden University, is one of the core conferences for researchers and practitioners in bibliometrics and research evaluation. Kathleen and Anton presented findings from the Meaningful Data Counts project, Stefanie gave a talk on developing an open science dashboard for biomedical institutions, while Marc-André’s and Leigh-Ann’s presented their respective doctoral and Master’s research projects in the same session on open access.
When interviewed about their experience at the conference, many lab members noted Leigh-Ann’s presentation on the oligopoly of academic publisher’s shift to open access publishing was a highlight. “I loved seeing people’s reaction when they realized that we have been paying billions in article processing charges to private for-profit publishers,'' says Marc-André Simard, a PhD student at the University of Montreal and ScholCommLab, and co-author of the new study. “It sparked some really interesting discussions among the people in the room and on Twitter.”
Despite many captivating talks and workshops at STI 2022, members of the lab were most excited to (re)connect in-person and chat over coffee. “The STI conference is always a pleasure to visit since it is really a meeting of friends,” says Isabella.
“After spending so much time working online with some colleagues, getting the opportunity to spend time in person with them makes you realize just how much we are missing out by working remotely,” says Marc-André. “It allowed me to have so many deep or informal ‘coffee break’ conversations with so many different people, including some that have led to new research projects and opportunities.”
Stefanie agrees that, while online talks and Zoom meetings are practical, the informal exchanges at in-person events are extremely valuable. “They remind me that there’s people behind the research and that makes our work so much more fun.”
Stefanie recalls attending her first STI conference in Vienna, Austria in 2008, when she had just started her PhD: “I still remember how overwhelming it was to take in all of those different, often very technical presentations and also to see those big authors for the first time whose papers I had been reading.”
That same conference 14 years ago paved the way for Stefanie to move to Canada in 2013. “I randomly ended up at the same table as Eric Archambault, at the time CEO of Science-Metrix, and Vincent Larivière, then still a PhD student. We stayed in touch over the years until they asked me to come work for them in Montreal after I had finished my PhD,” she says.
Fast forward to 2022—Stefanie attended the conference for the first time as a professor, reconnecting with her “people” at STI and introducing her fellow ScholCommLab members to the bibliometrics community. “The whole experience was amazing,” says Chantal Ripp, who had just started her PhD in the ScholCommLab with Stefanie. “I also met in person people that I had attended a summer school with, the Science of Science Summer School (S4), and that was great.”
“I met a lot of researchers who I had read and cited for many years which was pretty inspiring,” says Anton Ninkov, assistant professor at the University of Montreal. “There were so many intelligent and kind people there and it was just really great to swap our experiences and knowledge that came from all over the world,” he continues. “There were lots of ideas on how to approach the challenge of classification that I hadn’t considered before and inspired me in thinking about our lab’s own research on mapping different classification systems together.”
Kathleen Gregory, a postdoctoral research fellow at the ScholCommLab and the University of Vienna, was excited to attend the conference as part of a larger group. “I realized that I usually go to conferences by myself and meet people on my own,” she says. “Attending the conference as part of the ScholCommLab was a completely different experience. I had an automatic community to reflect on things with, have breakfast with, ask questions of, and use as a jumping off point to make new connections.”
As STI 2022 came to a close, each lab member felt recharged, inspired, and grateful for the quality time spent with familiar and new colleagues. “I came back from the conference so energized about my work, and in a new way that has somehow been missing in pandemic years. Now the challenge is to carry that energy through to the next conference,” says Kathleen.
The next International Conference on Science, Technology and Innovation Indicators (STI 2023) returns to where it originated and was first hosted in 1988—Leiden, the Netherlands. As Marc-André exclaims, “I’m already looking forward to the 2023 edition!”
For more on the conference, check out the STI 2022 website and program. Conference proceedings can be found on Zenodo. Read our preprint to learn more on open access fees paid by the global academic community to the oligopoly of five academic publishers, or check out the dataset on Zenodo.
Three questions with... Heather!
Our lab is growing! In our Three Questions series, we’re profiling each of our members and the amazing work they’re doing.
Today we're highlighting Heather Woods, a part-time professor in the Faculty of Education at the University of Ottawa, researcher and podcaster in Social Emotional Learning, and member of the ScholCommLab. In this post, she tells us about her work in the lab, learning through teaching, and offers one simple tip for embracing "learning opportunities" within and beyond academia.
Q#1 What are you working on at the lab?
I am working with Dr. Stefanie Haustein on the Metric Literacies project. I am collaborating with the team to conduct a literature review on different modalities for online learning videos. This review will guide which three or four formats we use to create our educational videos on metric literacies. We will then integrate a technology integration learning framework, called the Technological, Pedagogical and Content Knowledge Framework (TPACK), to enhance and optimize the video content and improve the viewer’s learning experience. We plan on using these videos to better understand and evaluate how individuals learn about metric literacies online.
Q#2 Tell us about a recent paper, presentation, or project you’re proud of.
I’ve recently had the opportunity to create and teach a graduate course on Social and Emotional Learning. It’s been a great opportunity to take my dissertation research and interests and explore it with groups of curious and thoughtful students and professionals. It enhanced my own learning about the topic, allowing me to explore how educators and professionals are able to make connections between research, policy, and practice within different settings.
Q#3 What’s the best (or worst) piece of advice you’ve ever received?
I think the best advice I’ve received within teaching, that can be applied to life and academia, is to be vulnerable and remember that you and your students are human. This informs the grace I show students, peers, colleagues, and others but also myself. I approach teaching and work with open communication, flexibility, and understanding. It’s really allowed my students to excel but also gives me space to experiment, explore, and make mistakes (or, as I like to call them, learning opportunities).
Read more about Heather on her website, research page, or find her on Twitter at @SEL_in_Action.