GEN AI Chatbot In Pharmacy Research
This research examines how pharmacy students use GenAI chatbots in their studies and professional development. It explores key challenges such as awareness gaps, reliability concerns, and limited curriculum integration. The study highlights the causes and effects of these issues, shedding light on the impact of AI-driven tools in pharmacy education.
Objective
The objective is to provide insights into the world of Generative AI Chatbots for research, highlighting their strengths, applications, and areas for improvement.
Through our research and synthesis of insights, we aim to offer solutions to address challenges and maximize the potential of Generative AI Chatbots in research.
Exploration of Existing AI Chat Bots useful to Students
Amazon Echo
1
Relevance
Source : https://aws.amazon.com/blogs/publicsector/building-a-multi-channel-qa-chatbot-at-saint-louis-university-using-the-open-source-qnabot/
The blog suggests leveraging Amazon chatbots to provide students with university information and proposes extending this to academic research tasks. This would allow students to verbally interact with chatbots for research materials, enhancing interactivity.
Harmony Mongoose
2
Relevance
Harmony mongoose is a good option for a chat bot as the universities can generate their own chatbots from the websites and implement it on the university websites. The chatbot is responsive and specially created for students.
Koala AI
3
Relevance
Koala is also one of the good examples of chatbots as it is very useful for creating essays and has a great interaction as per informed in the blog. Students and professors can definitely gain benefits of quick research and essays with the help of the chatbots
Primary Focus
In our project we will be specifically focusing on how Gen Al helps in research workflows, offering new possibilities for efficiency and productivity to pharmacy students and professors.
Gen AI is being used
by the Students
Feedback
Expectations
Solutions
Context of Research
Understand how the students are using the platform?
What are the advantages of using Gen Ai?
What pain points did the students face while using AI?
What are their expectations?
Knowledge Matrix
Affinity Mapping
Emily Smithson
Age
Location
Education
24
Philadelphia
Pharmacy Student
Attributes
Hardworking
Detail Oriented
Goal Oriented
Sia is a dedicated pharmacy student working on her cancer research project, which is due in 2 weeks. She relies on traditional research methods like books, journals, and articles to gather data. However, she struggles with time constraints and finding reliable sources for her project.
Goals
Limitations
Complete her research project within the 2-week deadline.
Search for relevant information to include in her research project.
Ensure her work is accurate, comprehensive, and meets academic standards.
Tight submission deadline for her project.
Difficulty in accessing or identifying credible, relevant research materials.
Limited efficiency using non-digital or outdated tools for research.
User Persona
User Journey Map
Feeling
Findings
Opportunities
Doing
Stages
Discovering AI Chatbots
Trying to Solve Problems
Seeking Verification
Last Attempt to Verify
Ending with Frustration
Sia learns about AI chatbots like Google Bard, ChatGPT, and Perplexity.ai.
Sia gives ChatGPT a try to solve problems and chemical equations.
Sia tries to verify solutions but finds the chatbot's response incorrect again.
Sia makes one last attempt to search for credible sources and calculations.
The chatbot provides no confirmed or accurate sources, leaving Sia irritated.
Neutral
Frustrated
Disappointed
Annoyed
Irritated
There are various AI Chatbots which helps students in their study
Chatbot provides inappropriate information.
Lacks references.
Cannot verify sources.
Solution remains incorrect.
Time is wasted retrying to search for answers.
Information is still inaccurate.
Sources are not confirmed.
Chatbot fails to provide any helpful or trusted response.
Educate users on the capabilities and limitations of AI chatbots.
Enhance chatbot accuracy for technical and academic queries.
Integrate proper references.
Include verifiable academic sources.
Reduce misinformation for critical queries.
Provide source links or references alongside answers.
Improve trust through transparency.
Enable real-time feedback for incorrect answers.
Partner with academic databases.
Problem Tree
The problem tree illustrates the challenges pharmacy students face in utilizing GenAI chatbots. It highlights root causes such as lack of awareness, reliability concerns, and limited integration in curricula, leading to effects like inconsistent adoption, misinformation risks, and reduced trust in AI-driven learning tools.
Four Main Problems
The four main problems encountered while using AI chatbots are lack of information, images, reference, and reliability. The responses provided by the chatbots often lack sufficient depth, making it difficult to obtain the detailed data required for research purposes. Additionally, the absence of visual aids, such as images or diagrams, hampers the understanding of complex concepts or technical problems. The lack of credible references further undermines the trustworthiness of the information, as it becomes challenging to verify the sources. Finally, the reliability of chatbots is a significant issue, as repeated inaccuracies and inconsistencies lead to frustration and wasted time, particularly for critical academic or professional tasks.
Lack of Information
No Images
No Reference
Unreliability
Lack of Information
I was searching for a hydrogel formulation for the eye for some of my drug, that time I won't be able to get enough information easily.
Root Cause
Lack of information
Chatbot does not have access to specific databases.
The images which I was trying to generate have never been the kind of accurate for me. I mean, no matter how many prompts I've had, reasonably bad quality images. So images has been a hindrance.
Root Cause
Issues with Images
Not able to upload or copy images or provide images as references
No images
It would be helpful if ChatGPT could provide references or citations for the information it generates. That way, I can verify the authenticity of the content.
Root Cause
No Reference
Chatbots do not have the ability to browse the internet or access to external sources
No Reference
For medical field, I would say accuracy of the information. At times, I have noticed that a lot of information wasn't completely accurate. And so the misinformation could be dangerous, especially when it's medical misinformation.
Unreliability
Unable to understand the technical names or topics
Reference to information is missing
Multiple sources of information
Information is provided without a proper source
Information is random and vague
No human intervention to verify correctness of data
Unreliability
Root Cause
Root Cause
Root Cause
Root Cause
Excerpts
37
Out of 50
Questions were deemed unfit for use
13
Deemed high risk for patient harm
Main Problem
Lack of information
Vague information
Does not provide information about research that was previously conducted for a topic
Doesn’t understand pharmacy specific terms like Rituximab
Unreliability
Source of information missing
Incorrect answers when calculating chemical equations
Relies on chatbot information 10% and on books and articles 90% of the time
Information about drugs needs to be accurate. Repercussions of wrong information in the pharmacy field could be life threatening
Not able to upload or copy images
Does not generate new images based on input provided
ChatGPT 3.5 does not provide images at all
No Images
AI needs to provide citations and sources of information
To provide proper answers which has scientific proof
No Reference
Through research and studies, a glaring issue has come to light:
Unreliability stands as the paramount challenge for pharmacy students when utilizing general AI chatbots for research purposes.
Key Insights 1
70%
60%
30%
10%
Out of 12 participants, 9 expressed concerns about the reliability of information obtained from ChatGPT. They stated that when using ChatGPT for research-related information, they must cross-check the provided details with credible sources such as research papers or articles to ensure accuracy and credibility.
Lack of Information
Unreliability
No Images
No Reference
Key Insights 2
In the article "ChatGPT and the Future of Digital Health," healthcare workers recognize ChatGPT as a valuable tool in digital health. However, they highlight the need to address concerns related to accuracy, reliability, and legal implications. Consequently, the article asserts that current versions of ChatGPT and similar chatbots should not be relied upon for diagnosis or treatment without the supervision of a qualified human expert.
Reliability
Accuracy
Legal Implications
Chat GPT and the Future of Digital Health: A Study on Healthcare Workers' Perceptions and Expectations
Key Insights 3
In the research paper "The Role of AI in Drug Discovery," it is noted that the data available to chatbots is often limited and may be of low quality or inconsistent. This limitation can significantly impact the accuracy and reliability of the results produced by these AI systems.
Data
(Low quality or Inconsistent)
Affect the Accuracy & Reliability of Data
Key Insights 4
Lastly, we observed that ChatGPT has significantly improved from its early versions, becoming an invaluable tool in advancing scientific research. However, it sometimes provides unsatisfactory or irrelevant responses when faced with ambiguous queries. Additionally, it has the potential to generate biased or inappropriate content, highlighting the need for continued refinement and ethical oversight.
Handling Ambiguity
Ethical Considerations
ChatGPT
Problem Statement
Pharmacy students encounter challenges using AI in research due to unreliable information lacking proper sources and citations, which undermines the credibility and trustworthiness of the data retrieved for their studies.
How Might We?
How might we enhance the accuracy of the chatbot data to ensure greater reliability in its responses?
Needs and Expectations
I just wish it would just like tell me where like what article is it from if we can pinpoint it to a point where I can just go look at it and try to understand the article as a whole.
I would like to see chatbots provide information with proper references, especially for complex topics like nanotechnology. Additionally, more detailed information on drugs and their mechanisms of action would be beneficial.
Recommendations
Maximizing the Potential of Gen AI Chatbot at Thomas Jefferson
1
Conducting training sessions
Seminars
Workshops
Why is this Important?
According to a Microsoft Work Trend Survey, as of March 2023,they surveyed 31,000 people in 31 countries.
82%
of leaders say employees they hire will need new skills to be prepared for the growth of AI.
60%
of people say they don’t currently have the right capabilities to get their work done.
‘’New Skills for a New Way of Working’’
Rapidly adjust to AI's integration in the workflow
Determine when to leverage a human capability instead of an AI capability
Evaluate content produced by AI
Evaluate AI fairness in decision-making
Direct AI with the right prompts
Ask AI the right questions
Determine when to leverage an AI capability instead of a human capability
Implementation plan
1
2
3
Identify target audience (faculty, staff, students) and their specific training needs.
Develop curriculum and training materials tailored to different user groups and skill levels.
Schedule training sessions and workshops at convenient times and locations.
Bridging Traditional Methods with Modern Techniques for Enhanced Reliability
2
Library
Chatbot
Our library is a trusted source of knowledge, offering a vast collection of expertly curated books, journals, and resources. By integrating our comprehensive database with the chatbot, we bring this reliability into the digital realm. Users can now access accurate and verified information directly from our library’s collection with ease and confidence.
Benefits and ROI
Benefits
ROI
Reliability: Combining traditional research methods with AI ensures more accurate responses.
Efficiency: Saves time by automating research processes and accessing information quickly.
Access to Knowledge: Integrates diverse resources for comprehensive information retrieval.
User Experience: Smooth interactions enhance overall satisfaction and engagement.
Cost Savings: Reduces labor hours and increases operational efficiency.
Productivity: Boosts output and academic achievements.
Enhanced Reputation: Demonstrates leadership in innovation and academic excellence.
Long-term Value: Sets the stage for ongoing innovation and growth.
Machine learning process
Existing ChatGPT Model
Data
Input
Ingredient
Training
Learning
Practice
Algorithm(s)
Rules
Recipe
Model
System
Chef
GPU
Hardware
Utensils
Prediction
Output
Cake
New ChatGPT Model Can be
Training
Learning
Practice
Algorithm(s)
Rules
Recipe
Model
System
Chef
GPU
Hardware
Utensils
Prediction
Output
Cake
Data
Ingredient
Input
OPEN AI
(Web scraping technique)
Library Books
Example
Carnegie Mellon University
Founded in 2017 by researchers seeking to solve the problem of how best to find relevant sources for a particular research topic, Keenious now has users from 145 different countries. CMU is the first university in the United States to implement Keenious, and the Libraries is thrilled to be able to incorporate this innovative tool into our information resource landscape.
Limitations
Possible Limitations
How can limitations can be addressed
Data Quality Issues
Data Quality Ensure the data used to train AI chatbots is accurate, up-to-date, and representative Issues
Offer training programs and support resources to staff and students involved in developing and using AI chatbots
Implement robust data protection measures, such as encryption and anonymization, to safeguard user privacy.
Technical Complexity
Privacy Concerns
Conclusion
In conclusion, while AI chatbots offer significant benefits such as time-saving and ease of use in pharmacy education research, there is a critical need to improve their reliability in providing accurate responses. Addressing this challenge will maximize the potential of AI chatbots as valuable tools in advancing research and learning in the field.
