EDGT983: Assessing And Evaluating In TESOL Environments

EDGT983: Assessing and Evaluating in TESOL Environments

Question

Answered

Your task is to evaluate a test or examination. The main aim of this task is to help you become familiar with the considerations and decisions involved in designing a test/exam/assessment instrument for NESB learners and whether these decisions have contributed to the validity, reliability or practicality of the instrument. You will draw on the knowledge developed throughout the subject (and your readings) to help you critically analyze and evaluate the test/examination/assessment instrument. Your essay will need to demonstrate a familiarity with the literature on testing in TESOL and the current issues facing the assessment procedure. 1. Write a critical evaluation of one of the following formal or semi-formal assessment procedures in TESOL. Choose one of the following assessment procedures/instruments:  a formal examination (e.g. TOEFL, TWE, IELTS, OPI, others)  a mainstream exam (e.g. Basic Skills Test, final examination, others)  an institutionally-developed assessment procedure (e.g. a placement test, diagnostic test, exit test, others) a teacher-made test for second language learners which is used in your institution 2. You will need to: (a) describe the nature of the assessment procedure (what context is the test used? Who are the targeted test takers? etc. Be as specific as possible as this will help with the evaluation) (b) use a set of criteria for evaluating the test (c) analyse the procedure in light of the criteria (d) report on its effectiveness or otherwise (e) include the chosen assessment procedure as an appendix To make this task manageable and allow you to focus on your critical evaluation of the procedure, you will use the set of criteria detailed by Brown & Abeywickrama (2019). GENERAL NOTES Some of you will be choosing a well-known standardized test such as IELTS or TOEFL. There are many studies and reports on these available in reliable websites. If you analyse one of these tests you will obviously need to do some background research into articles about IELTS and TOEFL. One of the problems with these articles is that most take a positive view and there is not much critical. You can then go into more depth on specific issues related to IELTS or TOEFL such as focusing on one of the macro skills OR on one or more of the above criteria for test design. Sources of information may also pose some difficulty, for example IELTS, as they are quite protective of their tests and procedures. You will need to exercise tact in order to get the relevant information you need. Some are choosing national exams such as the College Entrance Exam in China or the high school test in Japan. Again you should read the background research in journals such as JALT or other national journals. If you reference an article in Chinese or Japanese, put in the name of the journal article in the 12 | EDGT983 SUBJECT OUTLINE SPRING 2021 original language and add a translation in brackets. Write the name of the author in English transliteration. Some are choosing smaller more local exams. You will not have much primary source research – that is okay. However, you will be expected to do more analysis and investigation, including referring to related literature on the subject to support your claims or make your point. Important: Depth or Breadth – You can choose to go into detail, for example, analysing a reading test or you can focus more on broader contextual issues. Reading as much as you can on the topic(s) is essential; it adds not only to the breadth and depth of your critical evaluation, but more importantly to your knowledge. Essays reflect how much of the course readings their writers have done and understood. You will need to also refer to research (beyond the course notes) on things such as multiple choice, cloze or other techniques (communicative language testing, task-based approaches, or portfolios) if you are analysing these topics. Some guiding questions to help you develop your thoughts, ideas, and knowledge gained from the readings, reflections, and professional experiences, if any:  Are the test procedures practical? (Practicality) What in the assessment procedure demonstrate this? • Is the test reliable? (Test reliability/Rater reliability)  Does the procedure demonstrate content validity? (Content validity) Is the procedure face valid and “biased for test”? (Face validity) • Are the test tasks as authentic as possible? (Authenticity)