Charting the Digital Discourse: Understanding the Research and Instructional Value of Discursive Exchanges with AI

C. Sean Burns, PhD
Associate Professor
School of Information Science
College of Communication and Information
University of Kentucky

Commonwealth Computational Summit 2023: Artificial Intelligence

2023-10-16

cseanburns.net

Introduction

The integration of digital technologies is revolutionizing the instructional and research landscapes.

We have unprecedented opportunities for collaboration and intellectual inquiry.

Generative AI offers an additional tool to help with instruction, research, collaboration, and inquiry, especially with respect to an understudied area of science and scholarship:

problem-finding (especially for ill-defined problems)

Motivation

Why is a focus on problem-finding important?

Whole disciplines are dedicated to problem-solving but part of getting an education and part of being a great researcher is being able to ask good questions.

The role of outstanding scientists in influencing younger associates is repeatedly emphasized in the interviews with [Nobel] laureates. Almost invariably they lay great emphasis on the importance of problem-finding, not only problem-solving (Merton, 1968/1973, p. 453).

Merton, Robert K. (1968/1973). The Matthew Effect in Science. In N. W. Storer (Ed.), The Sociology of Science: Theoretical and Empirical Investigations (pp. 439–459). University of Chicago Press.

Motivation

Why is a focus on problem-finding important?

The formulation of a problem is often more essential than its solution, which may be merely a matter of mathematical or experimental skill (Einstein & Infeld, 1938, p. 92, as cited in Abdulla & Cramond, 2018, p. 198).

And:

Despite its theoretical and practical importance, it is surprising that PF [problem-finding] has received little attention … (Abdull & Cramond, 2018, p. 200).

Abdulla, A. M., & Cramond, B. (2018). The creative problem finding hierarchy: A suggested model for understanding problem finding. creativity. Theories – Research - Applications, 5(2), 197–229. https://doi.org/10.1515/ctra-2018-0019

Einstein, A., & Infeld, L. (1938). The evolution of physics. New York: Simon & Schuster.

Discursive

Generative AI, as a chat bot, offers a way to develop problem-finding methodologies using discursive reasoning.

I define and use the term discursive in two ways:

Socratic method: Problem-finding

For example, through dialogue, discussion, argumentation, and debate, the Socratic method can be used to:

Motivation

But what’s the problem I am responding to?

Instructor: Ask ChatGPT how to think about X problem.

Student asks one question and then stops.

Discursive, Learning, and Problem-Finding

Why discursive?

Discursive, Learning, and Problem-Finding

Why discursive?

To the extent that knowledge acquisition and academic performance are the goals, teacher-controlled methods of students’ active learning are superior–especially for younger students and in well-structured subject domains. In contrast, if the goal is to help students to “learn to become independent thinkers and learners rather than simply become able to perform basic computations and retrieve simple knowledge and facts” […], then methods in which students have opportunities to learn more independently, to identify, define, and solve problems in different ways, and to use acquired insights to overcome other tasks are more likely to be successful (Weinert & Helmke, 1995, p. 141).

Weinert, F., & Helmke, A. (1995). Learning from wise Mother Nature or Big Brother Instructor: The wrong choice as seen from an educational perspective. Educational Psychologist, 30. doi: https://doi.org/10.1207/S15326985EP3003_4

Problem-Finding

As an example, I used ChatGPT to problem-find a framework for problem-finding.

Let’s begin.

ChatGPT: Articulating a Problem

Interactions with ChatGPT have been abbreviated.

I asked: What are some approaches to articulating a problem that can be applied in instruction and research?

ChatGPT: Narrowing to a Research Question

I followed up with: What are some approaches to refining a research question?

ChatGPT: Consultation and Feedback Approach

I focused on one answer: How might consultation and feedback help refine a research question?

ChatGPT: Engagement

I asked for more refinement: What are methods and strategies to maximize the effectiveness of a conversation or brainstorming session to refine a research question?

ChatGPT: Engagement (cont.)

ChatGPT: Develop a Framework Based on Prior Response

Based on the prior responses, I asked ChatGPT to develop a framework and to present it in tabular form.

ChatGPT: Develop a Framework Based on Prior Response

Phase Strategy How to Engage Generative AI (e.g., ChatGPT)
Preparation Background Research Agenda
  • Feed the AI summaries or key points from existing literature to set the context.
  • Clearly state your objectives for the AI session.
Structure Time Management Moderator
  • Decide in advance how long you’ll engage with the AI and what topics you’ll cover.
  • You’ll act as the moderator, guiding the AI through the discussion topics you’ve outlined.
Inclusivity Diverse Perspectives Equal Participation
  • While AI doesn’t offer diverse “personal” perspectives, it can generate a range of ideas based on its training data.
  • Ensure that all generated ideas are considered.
Techniques SWOT Analysis Mind Mapping Probing Questions
  • Ask the AI to list potential Strengths, Weaknesses, Opportunities, and Threats for your initial research question.
  • Use the AI to generate related terms or questions, which you can then organize into a mind map.
  • Use the AI to generate probing questions that could help refine your research question further.
Documentation Real-time Notes Whiteboard or Digital Tools
  • Keep a record of the AI’s suggestions and your own thoughts as you go along.
  • Copy and paste the AI’s suggestions into a digital whiteboard or note-taking app.
Critical Thinking Devil’s Advocate Ranking and Voting
  • Ask the AI to generate counterarguments or criticisms of your initial research question.
  • Ask the AI to generate a list of refined questions, which you can then rank or vote on yourself or with your team.
Ethical and Practical Considerations Ethical Review Feasibility Check
  • While the AI may not provide ethical guidance, it can generate a list of ethical considerations you should be aware of.
  • Ask the AI for potential challenges or limitations related to your research question.
Follow-up Action Items Feedback Loop
  • Use the AI to help list next steps based on the discussion.
  • Return to the AI for additional sessions as you refine your question further.
Reflection Meeting Summary
  • After the session, review the AI-generated content and your own notes to create a summary.

ChatGPT: Create the Study

I asked ChatGPT to write up a description of a study that could test how the framework works as a problem-finding strategy.

Study Title

“AI-Enabled Discursive Exchanges and Their Influence on Research Conceptualization, Methodology, and Epistemic Limitations in Undergraduate STEM Research”

Objectives

  1. To investigate how AI-enabled discursive exchanges influence the conceptualization and elaboration of research topics.
  2. To identify patterns in these discussions that can be harnessed to refine research methodologies.
  3. To explore the limitations and challenges posed by such interactions, especially concerning the epistemic nature of AI.

ChatGPT: Create the Study

Participants

Undergraduate students enrolled in STEM programs.

Methodology

Phase 1: Preparation

Phase 2: Structure

ChatGPT: Create the Study

Phase 3: Inclusivity

Phase 4: Techniques

ChatGPT: Create the Study

Phase 5: Documentation

Phase 6: Critical Thinking

ChatGPT: Create the Study

Phase 7: Ethical and Practical Considerations

Phase 8: Follow-up

Phase 9: Reflection

ChatGPT: Create the Study

Data Collection and Analysis

Expected Outcomes

ChatGPT: Conclusion

Finally, I asked ChatGPT to summarize and provide some key takeaways from our session:

Key Takeaways

Conclusion: Problem-Finding