Determining the effectiveness of online learning interventions is essential to continuous online course improvement. Action research offers an exciting model for instructors to meet this challenge by enabling them to quickly probe for sensible solutions to pedagogical problems.
Action research is a systematic investigative process performed by the practitioner either during an online course or at the end, and can involve quantitative and qualitative methodologies. It can provide meaningful information on important topics such as:
- teaching strategy efficacy
- instructor-student interactions
- student-student interactions
- student-technology interactions
- student performance
- student achievement
- career self-efficacy
- student engagement
There is no definitive book on implementing action research methods in the online science, technology, engineering, and mathematics (STEM) classroom, but Action Research Methods: Plain and Simple, edited by Dr. Sheri Klein, offers both theoretical and practical information for implementing action research methods. Action research provides a sustainable approach to produce beneficial longitudinal data exploring student perceptions and cognitive outcomes and may lead to new pedagogical strategies that enhance enrollment, retention, and academic success. The benefits of action research in traditional science education courses are well documented. If properly conducted in online courses, it can provide immense benefits to online teaching and learning outcomes as well (Feldman & Capobianco, 2000).
In general, canonical action research approaches consist of the following six cyclical phases:
1. Determine the nature of the investigation.
Research questions must address issues germane to course objectives, departmental competencies, and student learning outcomes. Science education research questions could be constructed to study online science laboratory methods, discussion board usage, online science projects, scientific writing assignments, virtual presentations and experimental demonstrations, problem-solving skills, scientific reasoning skills, data analysis skills, scientific method comprehension, student knowledge, and student understanding of classical and contemporary scientific principles.
Research questions should be specific and measurable. Questions must clearly address the treatment population, particular intervention, and desired aim of the research project. Alternatively, research questions could be designed simply to elicit ideas from students that can be implemented immediately to improve the overall learning experience. During this first stage it is also prudent to engage in a review of the research literature to determine if your question has already been addressed by other researchers or to review similar studies to determine appropriate research techniques and instructional strategies that may be employed in your current investigation.
2. Obtain Institutional Review Board (IRB) approval.
The IRB is typically an institutional committee tasked with reviewing research protocols, qualitative and quantitative surveys, data analysis procedures, and dissemination strategies to confirm that the entire research pathway is aligned with stringent federal policies designed to protect human subjects. Typically, researchers must obtain appropriate certifications before engaging in research activities on campus. A series of widely used Web-based certification modules can be accessed at the Collaborative Institutional Training Initiative website (www.citiprogram.org). A brief discussion of your proposed action research activities with your IRB or similar institutional compliance officer may be instrumental in determining the correct human subjects protection process and necessary certifications required for your institution. Pedagogical researchers should maintain ongoing IRB approval for each study to negate potential subsequent legal actions and to ensure that the highest ethical standards of research are being met.
3. Conduct the study.
In this phase, an intervention or pedagogical method is implemented within a specified time frame to address the specific research question. Selection of the treatment is particularly important. Online faculty must ensure that the treatment chosen will effectively elicit the desired information needed to inform course activities.
Raubenheimer and Myka (2005) performed a study designed to determine the effects of laboratory exercise changes on student learning. In this study, the researchers modified existing zoology laboratory modules to focus on higher-order cognitive tasks. After the students completed the modified laboratories, the researchers administered a questionnaire that assessed student perceptions of learning gains and levels of enjoyment. They found that there were components of the modified lab from which students derived a high degree of educational value and that there were components from which students derived the least amount of education value.
4. Collect the data.
Sample size is not a major consideration for a successful research experience. Data can be collected from one or 100 students and still elicit the same beneficial effects—improve online teaching methods and knowledge acquisition. There is a wide variety of existing qualitative and quantitative surveys that can be located by searching the ERIC database (www.eric.ed.gov). Although the type of questioning mechanisms will differ based on the specific issue explored, the following valid and reliable quantitative surveys are readily available: Survey of Academic Orientations, Academic Motivation Scale, Institutional Integration Scale, Career Decision Self-Efficacy Scale, Learning Self-Regulation Questionnaire, and the Scientific Reasoning Test.
Alternatively, you can create your own questionnaires based on your unique online course experiences. The use of pre- and post-measurements are especially beneficial and will quickly provide evidence of a successful intervention. For example, if you want to determine the effects of an online STEM career development module on students’ career self-efficacy, you could administer the Career Decision Self-Efficacy Scale early in the semester (pre-test), implement an online career development module, and then re-administer the scale later in the semester (post-test) to determine the effects of the online instructional strategy on career self-efficacy.
5. Analyze the data.
Analysis of the data resulting from action research projects can involve many formal and informal processes. Unlike traditional forms of research, action research analytic procedures can include examination of scores on a carefully designed rubric, quiz, or test, and can include analysis of student activity data derived from a learning management system. Descriptive and inferential statistical analysis can be used, depending on the nature of the research; however, in action research those rigorous methods of analysis are not required. The meticulous inspection of data to observe patterns or trends are paramount tasks in this phase. Consider using tables, bar graphs, and pie charts to present the data to aid in the analysis of the results. The ultimate goal of the analysis phase is to determine the impact of the pedagogical treatment on online learning and teaching.
6. Reflect, implement, and disseminate.
During the reflection phase, contemplate the significance, meaning, and implications of the results—either alone or in a collaborative team. Allow the research findings to guide specific enhancements to the online course that accurately address the original issue or research question. In the implementation phase, I encourage you to redesign or reconsider specific online course components to support course learning outcomes. Following the reflection and implementation phases, new research questions typically arise that may lead to additional research investigations and the return to a preceding step in the action research process. Results of the action research project can be disseminated in a variety of ways, including publication in scholarly journals, at faculty meetings, via the Internet, or at workshops, seminars, or academic conferences.
I recommend using SurveyMonkey to create surveys for use in action research. SurveyMonkey (www.surveymonkey.com) is a Web-based software program that allows online instructors the opportunity to create user-friendly questionnaires. The use of SurveyMonkey is free when preparing surveys containing 10 questions or fewer.
Conclusion
Action research plans can provide online instructors with a transformative approach to improve their overall performance and outcomes. Research methodologies can be conducted on a group of students or with individual students. As with any science education investigation, the development of a robust research question is paramount and guides every aspect of the research investigation, including, but not limited to, questionnaire selection, research participants, sample size, data collection procedures, and statistical manipulations. Phenomenological data from action research initiatives will allow online STEM instructors the opportunity to make prudent judgments to improve teaching and learning and may subsequently lead to the dissemination of beneficial intellectual resources for a variety of stakeholders.
References
Feldman, A. & Capobianco, B. (2000). Action research in science education. Retrieved from ERIC database. (ED463944)
Klein, S. (2012). Action research methods: Plain and simple. New York, NY: Palgrave Macmillan.
Raubenheimer, C. D. & Myka, J. L. (2005). Using action research to improve teaching and student learning in college. Journal of College Science Teaching, 34, 12-16.
Acknowledgements
This work was supported by a grant funded by the National Science Foundation (HRD-1332555).
Lawrence O. Flowers is an assistant professor of microbiology at Fayetteville State University.
