Techniques in Educational Evaluation
Code
EDA 641
Short Course Description
This course provided tools and practice in the evaluation of an educational program. An IRB-approved project was implemented with two different program clients, and students worked in groups to gather preliminary survey data for continuation of the program evaluation after the course was complete. Qualitative research tools such as client interviewing, program theory formation, research question design, and qualitative survey design and evaluation were implemented.
Greatest Course Takeaways
This course was a great connector between my Introduction to Qualitative Research (EDU 603) and my Instructional Design and Development (IDE 631/632) courses. Why? Because it required us to utilize qualitative research techniques to address the concerns of a client, which can commonly occur when designing instruction for others (as seen in the course product for IDE 631). This class provided the opportunity to participate in real, IRB-approved research in an academic environment and taught us how to interact with both clients and research audiences/participants. One of my greatest takeaways from the course is the ability to work with the client to narrow down their interest - which becomes our research question to answer - and figuring out ways to obtain information that answers those questions in an unbiased way. The wording of questions utilized on surveys and having these questions reviewed by outside eyes was a very important aspect that I took away from this course as well. You must ensure that your questions and the answers or options you provide will result in information that can be used as evidence and not just data, and that this information is gathered without bias towards one outcome or the other. While this class was abruptly interrupted by COVID mid-way through, it was still very useful to me. It also gave me insight into the IRB process to prepare me for research work in the future should I need to utilize human subjects for my studies in education and instructional design.
Course Product Description
The course product for this portfolio was the final project for the class. This was a multipart assignment in which students worked in groups to evaluate an educational program. We were provided with descriptions of the clients and their general questions about their programs to select from. We then met with the clients to learn more about the program and their goals for this evaluation to translate into research questions. From this information, our report included a program description with key information about the program and target audience, the scope of our evaluation and questions that were were asking, the methods and designs we were to use in the evaluation, excerpts of the data or data samples utilized in our analysis, and the evaluation and interpretation of the results for the client to use moving forward.
My group worked on examining the LL.M. Program at Syracuse University. We employed surveys to and utilized interviews of current students to get a better understanding of their perception of the program. Particularly, we wanted to see if they were satisfied with various aspects of the program, what the program was doing well, and what could be improved. An example of this was performance of the faculty from the students' perspectives. We also examined which services of the program - such as the mentoring services and extracurricular activities - students utilized and what their perspective of these resources were.
Click Here to See the Course Product for this Course and Others in the Portfolio.
Selection of Product for Portfolio
This product was chosen for the portfolio because it is a great example and outline of what an evaluation project for an educational program would require. If I aim to work with curriculum in the future, I need to be able to evaluate educational programs like the one in the course product. It was a great group effort that we all contributed to and worked hard towards satisfying. It also provides evidence of working directly with a client in a real evaluation and study, not just one that is proposed based on preliminary data collection or a lesson that is created with certain educational concepts and topics but never implemented. This product was the result of working directly with members of the LL.M. program to get to the root of the client's curiosities and questions.
This course project also provided an opportunity to combine the qualitative research skills I had learned in the program thus far with quantitative analysis skills to reach a conclusion in the findings. This was an important step for me because I had still been viewing the qualitative and quantitative research tools as different worlds instead of one in which both could be combined for powerful analysis. This course product gave me the opportunity to meld the two. The findings generated from my thematic analysis and statistical comparisons discovered discrepancies in the student experience in the educational program that was influenced by gender and age of the students. This was a surprise to the clients, and it provided valuable information for them as they worked on evaluating and modifying their educational program for future students. This was very satisfying to see the culmination of these different research skills into a project that could enact actual change at the collegiate classroom level, something that I aspire to do in my work as a future professor.
Contribution of Product to Overall Program
I alluded to this earlier and throughout other portions of my reflections on my coursework and course products, but one of the major reasons for enrolling in this Teaching and Curriculum program was to become a better educator. This is not possible with teaching skills alone. One must also be able to evaluate their own performance and the existing academic structures in place to determine if students are being adequately served and supported, or if change is necessary. This course product shows exactly that: the capability of examining an educational program with guided research questions and both qualitative and quantitative research skills to collect data and interpret information for meaningful results and potential change for the better.
The more I learned about learning and instructional design, the more I became fascinated with curriculum as well. Not only did I want to change the way we teach students college science and engineering in the classroom, but I wanted to reform how we created and presented an entire engineering curriculum or program. This was inspired by my findings in the course product for SCE 718 (Curriculum Problems in Science) as well as this course. Working on this course project developed and refined my research and evaluation skills and provided a framework that I can utilize in the future as I work to improve upon the educational experiences of my students throughout their college tenure. This course product was a very valuable experience for me in that regard. If I am to strive for change, I need to be able to evaluate what is already in place, and that is not possible without course products such as this one that gave me practice in these research skills. I also enjoyed that it involved a collegiate-level program. While it wasn't science related, it still provided insight into the way a college program operates and some of the factors that administrators and faculty that run these programs find important when evaluating its effectiveness. This course product - along with many of the others - unknowingly influenced the development of my final course product in this program (the Curriculum Design Model in IDE 632: Instructional Design and Development II).