Application of AI to overcome scientific information overload Novel uses of natural language processing for more meaningful science
in response to a prompt. As of January 22, 2023, ChatGPT has also been listed on PubMed as an author on 2 publications and 2 pre-print articles. [6,7] A major limitation of language models like ChatGPT that provide text answers to queries, is hallucination ; answers generated may be mostly correct but are not always so, which can result in inaccuracies and the creation of misinformation. The models are often not able to explain their answer as the computations performed to generate the answer are different from human reasoning (this can be described as a “black box” where there is no transparency in how AI algorithms process information). [8] Hence, there is emerging research in Explainable AI , which is a set of processes or methods that allow human users to comprehend and trust the results and output created by AI systems.[9] There is a lot of cause for excitement given the progress in this subfield of AI, but also a need for prudence and general education. Regardless of the platform or use case, clinicians should understand the data used to train these AI models and be aware that language and even data itself can contain biases (e.g., in race, ethnicity, and gender). Frequency and social bias towards popularity can also occur. Feeding biased data into AI algorithms can produce systemically biased outputs. [8] Project INSIDE-PC AI is the catalyst to disrupt information overload and lead towards more meaningful science. The INSIDE-PC ( artificial IN telligence to Support I nformed DE cision making in P rostate C ancer ) project explores novel uses of natural language processing and AI to derive insights and answer queries from the complex universe of scientific literature and clinical trial databases. With this technology, content and meaning are extracted from searches inside publications and clinical trial databases to identify not just the presence of key terms but the context in which these terms are being used (called “semantics”). The semantic analysis and AI framework used to help experts delve INSIDE publications for information related to a specific query can also be repurposed to address other future questions of scientific interest (Figure 4). INSIDE-PC tackles one of the most complex issues within the literature for oncologists and urologists who treat genitourinary cancers – specifically the issue of therapeutic sequencing. This includes consideration of how to sequence classes of drugs such as NHTs, PARP inhibitors and emerging agents like radioligand therapies. Building from work previously published in BJUI by Stenzl et al. [8], AI algorithms are used to emulate how human clinicians extract sequences for advanced prostate cancer treatment from scientific publications in order to answer the question: what are the outcomes when using drug X (or drug class X) followed by drug Y (or drug class Y) ? A sequence (A->B) is a temporal relationship (e.g., patients who were initially treated with treatment A and subsequently treated with B) that can be challenging to extract and interpret from the literature. Ultimately, INSIDE-PC helps to address this challenge as users can filter their searches for a specific treatment sequence and extract the associated scientific publications that
Jenny Ghith Pfizer Inc. Collegeville, PA
On behalf of the INSIDE-PC* Working Group
*INSIDE-PC (artificial IN telligence to S upport I nformed DE cision making in Prostate Cancer) Working Group: Arnulf Stenzl, Cora Sternberg, Andrew Armstrong, Andrea Sboner, James N’Dow, Lucile Serfass, Christopher Bland and Bob Schijvenaars The scale of data and scientific literature available today is surpassing metrics predicted only a few years back. Over 3,200 clinical trials in urologic cancers are ongoing as of January 2023. [1] Approximately 20,000 cancer-related articles were accepted in 10 high impact urology journals in 2022 (Figure 1). The COVID-19 pandemic only accelerated data availability, as researchers set new standards and operated within a short time frame to produce findings, vaccines, and treatments [2], with articles being published at a rate of up to 14,000 each month. [3] In addition to surges in scientific data, digital advances are enabling substantive increases in patient-level data through electronic health records, wearables, and mobile apps. What is information overload? These increases in knowledge are cause for celebration but are also a double-edged sword. In a 2021 internal survey of 150 US and EU physicians, 60% of oncologists and 80% of urologists treating patients reported they were not able to easily find answers in the literature to complex questions on treatment of prostate cancer (Figure 2). Amongst physicians, information overload can lead to practice inconsistency, scepticism, uncertainty, distrust of medical evidence, and lack of awareness or adherence to recommendations from Societies and Committees. Moreover, access to an excess of complex and conflicting cancer-related information can cause worry and confusion among patients, who are increasingly turning to search engines like Google and social media for their health information.[4,5] How can artificial intelligence (AI) and language models help? What are the challenges? AI is being used in oncology for tasks such as diagnostics, imaging, drug discovery, and cancer prognostics, but has been more of a nascent field in literature mining and writing academic papers. AI can help clinicians search and analyse literature, extract useful information, identify knowledge gaps, and stay updated on new developments in the field. Machine learning development speed is staggering in multiple areas, including text analysis, speech, images, and combinations of these. Interest in this area is only increasing. For example, in late 2022 ChatGPT was released by OpenAI and gained over one million users within 5 days. ChatGPT is a state-of-the-art language model that uses neural network architecture to generate human-like text
Figure 3: Progress in AI language models and image generation
Figure 4: The premise of INSIDE-PC*
5. Develop tools to automatically summarize text and write short articles For these reasons, it is incumbent upon clinicians to stay informed and understand the strengths and limitations of emerging AI technologies. It will help the healthcare community to obtain more personalized information and achieve better outcomes for patients. In fact, in many ways AI is no longer emerging—it is already here today. References 1. Digital Science. (2018) Dimensions [Software] available from https://app.dimensions.ai. Accessed on Jan 9, 2023, under license agreement. tinyurl. com/22d8yumn. 2. Else, H. How a torrent of COVID science changed research publishing — in seven charts. Nature 588, 553–553 (2020). doi: 10.1038/d41586-020-03564-y. 3. Pubmed search. Accessed January 2023. tinyurl. com/yntb22mw. 4. Jensen, J. D. et al. Health information seeking and scanning among US adults aged 50–75 years: Testing a key postulate of the information overload model. Health Inform J 23, 96–108 (2017). doi: 10.1177/1460458215627290. 5. Reifegerste, D. et al. Understanding the Pathway of Cancer Information Seeking: Cancer Information Services as a Supplement to Information from Other Sources. J Cancer Educ 1–10 (2021). doi: 10.1007/ s13187-021-02095-y 6. ChatGPT & Zhavoronkov, A. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 9, 82–84 (2022). doi: : 10.18632/oncoscience.571 7. O’Connor, S. & ChatGPT. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Educ Pract 66, 103537 (2023). doi: 10.1016/j.nepr.2022.103537 8. Stenzl, A. et al. Application of Artificial Intelligence to Overcome Clinical Information Overload in Urologic Cancer. BJU Int, 130: 291-300 (2021). doi: 10.1111/ bju.15662 9. Khan, M. S. et al. Explainable AI: A Neurally-Inspired Decision Stack Framework. Biomimetics 7, 127 (2022). doi: 10.3390/biomimetics7030127.
presented with its validation during the EAU Congress on March 13, 2023, during the thematic session “Is Artificial Intelligence (AI) in Urology Ready for Prime Time?” . Searching the scientific literature and clinical guidelines for specific therapeutic sequences of key interest in traditional ways (e.g., via searches on PubMed) is difficult because of the heterogeneity in how sequences are described, the time-consuming and complex nature of searching, the similarity of terms related to “genetic sequencing”, and the complexities in how clinical outcomes are reported. At worst, key data sets that impact clinical decision making can be missed or misinterpreted. INSIDE helps clinicians to overcome these challenges and extract the most relevant information in personalized ways. To evaluate the utility of INSIDE-PC, key search capabilities are benchmarked against PubMed queries and index papers chosen by subject matter experts. Conclusions Advances in AI can address information overload and misinformation by saving time and effort as well as capturing meaning and context that improves accuracy. This is demonstrated by INSIDE-PC from the INSIDE framework. The INSIDE model can be adapted for future queries with the goal of enabling users to ask questions about therapeutic sequencing in other tumour types – or more broadly (and importantly) – to ask other questions of relevance for prostate cancer itself or different cancers entirely.
reported its use. The full text of scientific documents are searched for the sequence, and each
Ultimately the technology can be used to 1. Extract additional information and
relationships between terms (e.g., identifying adverse events or subgroups and how they are impacted by treatments) 2. Develop direct question & answering systems that deliver precise answers to queries and can handle simple requests and tasks (vs. traditional search engines, where users must extract answers themselves) 3. Advance more targeted data extraction (e.g., ingest trial and real-world evidence data for further analysis or hypothesis generation) 4. Develop “living SLRs” and Guidelines (that allow Committees to surface the most relevant information more quickly and accurately)
document is analysed for
sentences that are relevant based on the context, and users are provided with the ability to sort on parameters including the level of evidence and type of outcome. The dashboard is under development and will be
Figure 1: Growth in urology journal publications
Monday 13 March, 10:50 - 11:00 Thematic Session: Is artificial intelligence (AI) in urology ready for prime time? Pink Area, Coral 4
Figure 2: Information overload amongst urologists and oncologists
European Urology Today
February/March 2023
17
Powered by FlippingBook