Using Internet-based language testing capacity to the private sector

Abstract

Language testing has a large number of commercial applications in both the institutional and the private sectors. Some jobs in the health services sector or the public services sector require foreign language skills and these skills require continuous and efficient language assessments. Based on an experience developed through the cooperation of Antwerp University and the Polytechnic University of Valencia, a language testing platform was used to produce simple and multi-language skills assessments for a publishing company. Results indicate that academic low stakes testing platforms can replace efficiently the use of larger tests or costly language testing companies that either are expensive, or just do not tailor the tests according to their customer specific needs properly. Since academic institutions work on a non-profit basis, the revenues can be used for further research or improving their current developments.

Keywords

testing, academic, commercial, clients

Background

A large number of private companies demand a second (or subsequent) language to undertake certain jobs in a country or abroad: nurses living and working abroad, for example international telephone operators, international executives or business people (Gross, 2004; Clarke, 1999), and health professionals (Moreno et al., 2007). Thus, language assessments are absolutely necessary for many private companies (Chahoulb & Tarone, 1996). This demand has been growing over the last 10 years due to the mobility of working labor as well as the increasing demand for certain jobs in specified European and American countries (Maiworm & Teichler, 1996), such as those related to health. Private companies' capacity to communicate effectively with their workers is also necessary and crucial to the company's organization and quality service delivery. This is especially true when potential clients are culturally and linguistically diverse. Therefore, private companies rely on costly and not always efficient tests to assess their workers. Overall, the most commonly accepted tests worldwide are IELTS, TOEFL, BULATS (The Business Language Testing Service), and OET (Occupational English Test) (O'Neill, 2007; Iwashita & Gove, 2003). However, these tests may not be able to cope with the companies' or institutions' needs especially in the health sector (McCabe et al., 2008). They also usually involve a more or less lengthy process that includes registration, test delivery, marking, and return of the results. The process is usually expensive, inefficient and by the time the results are returned one (or some) prospective workers may have decided to look for another job; as a result, the company loses its money and part of the selection process is a waste of time.

On the other hand, academic centers worldwide have the resources to develop especially tailored courses and assessments for their own students. This is not just for languages but also for general core subjects. Generally, these assessments are low-stakes tests and permit assessments to be made within a few days, from the time they are programmed to the moment when the results are returned (Rouse & Krueger, 2004). In general, these low stakes tests require different security systems and, overall, are tailored to each specific case. Additionally, their use is not homogenous all through the year. There are usually peaks and periods with limited use; for instance, in the European context, May-June may be a peak, but February-March is a period with minimal use.

The Polytechnic University of Valencia has developed an oral-written flexible testing system for oral evaluation that has been described in some academic papers and is currently developing a second version. On recent trials, it was seen that although the system was allowed for the whole 25,000 student population under demand, only a few professors requested its use. Thus the question was, why not offer it to a private company? As a result, the testing platform was offered to a publishing company twice a month for a free-of-charge teacher training course on the prospective University Entrance Examination language test (details below). 25 teachers used the academic platform.

Goals

The main goal of this case study was to see whether the platform could cope with the different needs required of it, and whether it could function as an efficient and low cost tool for taking tests by these teachers. The secondary objectives were as follows:

  1. Analyzing the platform's flexibility and adaptability to specific demands (suggested by the private company),
  2. Suggesting a pattern of use for private companies,
  3. Addressing guidelines of good practices in using academic software for commercial purposes, and
  4. Suggesting ways to implement academic language testing platforms for commercial use.

Subjects

The teacher trainees took a number of trial tests in order to observe and develop a special sensibility towards computer-assisted language testing. The only incentive that they received was a course certificate and a deeper understanding of computer-based language testing processes. However, they always knew that the University Entrance Examination would differ to the mock tests they were taking, in length and difficulty. However, since the main goal of this paper is related to the flexible use of language testing platforms, we will no go any further into the subjects' specifics.

Description of the activity

During the academic course of 2007-2008, the Universidad Politecnica de Valencia (UPV) finished a prototype of an online test for foreign languages that included both oral and written sections. The platform was designed to evaluate:During the academic course of 2007-2008, the Universidad Politecnica de Valencia (UPV) finished a prototype of an online test for foreign languages that included both oral and written sections. The platform was designed to evaluate:During the academic course of 2007-2008, the Universidad Politecnica de Valencia (UPV) finished a prototype of an online test for foreign languages that included both oral and written sections. The platform was designed to evaluate:During the academic course of 2007-2008, the Universidad Politecnica de Valencia (UPV) finished a prototype of an online test for foreign languages that included both oral and written sections. The platform was designed to evaluate:

  1. Multiple choice and objective responses,
  2. Listening tasks prompted either by listening or listening and watching prompts and input,
  3. Writing tasks either by listening or listening and watching prompts and input,
  4. Reading sections, and
  5. Speaking, either by listening, or listening and watching prompts and input.
Since the use given to that platform until that moment had been merely for research purposes, it was thought that some professional use could be added to regular use in the academic environment. At that time, Heinemann Iberia was just about to begin a teacher training course in language testing. So the publishing company suggested using the platform and the UPV considered the possibility to use it for testing for commercial purposes. Since the relationship between the Universidad Politecnica de Valencia and Antwerp is a long standing one- first through the Didascalia Institute and today through Linguapolis- , the UPV requested advice from them. As a consequence, the Belgian institution recommended delivering the tests as follows:

Objective Initial assessment: Teachers would experience the web based tool Individual skills, grammar and vocabulary assessment Final assessment
Computer platform use Full exam was delivered Separate types of exam were delivered:

Multiple choice; Written-listening; written-reading; Reading-Listening; and so on
Full exam was delivered (four skills plus multiple choice)
Approximate dates January 2008 February-April 2008 June 2008


Teachers took either one-skill tests as well as integrated tasks tests. Results for the 15 teachers were delivered in two days because the raters were volunteers. However, these tests (either written or oral) could be rated almost on the same day, if professional raters were used. Additionally, multiple choice tasks were self-corrected. The publishing company provided the contents for the test so the takers had to take both general and very specific tests.

Customer satisfaction

Being the academic testing platform, we expected the teachers to feel at ease from the beginning. Thus, after the first session a twelve-item questionnaire was distributed. Results indicate that teachers were rather satisfied with use given to the platform as a whole. The teachers stated that the tutorial was one of the major achievements. As a consequence, it is important that if potential customers are not familiarized with the platform, at least to include a clear and user friendly tutorial, and if the computer platform is not as straightforward as possible in its use, adaptive to its purpose and goals, the client may not understand that the platform is currently measuring what is intended to (face validity).

Evaluative comments and conclusions

The activity was considered positively. Teachers valued saving time in their assessments and also were happy because each individual activity and task corresponded exactly to what was necessary and expected in each moment and course stage. The older teachers acknowledged the difficulty of taking online tests when their computer skills were limited but also recognized that handling the computer tool had been simpler than they thought at the beginning. Eventually, most of them got used to the computer tool and believed they performed almost free of anxiety and computer presence awareness. Most of them also believed that taking the test would be appropriate in other professions and professional situations.

Teachers or prospective test takers would have to face different activities according to the hiring company. The great advance as opposed to the standardized tests is that, like in the case hereby described, the company states the conditions and suggests the kind of materials to be included in the test. Furthermore, automatic rating devices (like e-raterTM or a speech recognition system may be designed to a considerably lower cost than only 6 years ago). Thus, tailoring language tests may be faster, less costly and more efficient that hiring large companies like ETS or the Cambridge Board of Examinations. Besides, tests can be tailored to the stakeholders' needs because the hiring company may suggest what it needs to be observed, the taker only has to perform tasks which are relevant to the job (or situation), the delivery institution only devotes those resources that are required (not the whole set of raters and test administrators).

Advice to other institutions and test deliverers

Language testing must be flexible and adaptable to its needs. Relying on fixed testing structures may be a major drawback when designing online language tests. Reducing costs and developing multimodal platforms is a must in language testing. Good academic developments offer great opportunities to benefit all the stakeholders especially the designers and central administrators. On the other hand, these platforms offer tailored responses to the most specific demands. Designing platforms specifically aimed to certain tests may be valuable and certainly a solution for large organizations such as ETS or the Instituto Cervantes (to mention just two) but low stakes tests and platforms certainly are less costly and still produce significant revenues. The advantage of these platforms is their flexibility and that fact that private companies do not need highly secure platforms (like ETS or the typical University Entrance Examination in Spain), but rather efficient and low cost examination systems. Additionally, the flexibility is adding value, because these platforms can also work as a part of language courses platforms and, in this way, reduce the costs of the academic or commercial courses through the revenues produced by the platforms' commercial use.

Note

I would like to express my gratitude to Christine Engelen (Linguapolis, Antwerpen University) for pushing me (literally) to write this paper and giving me her support and ideas and also to the Ministry of Education of Spain for supporting the development and implementation of the testing platform hereby mentioned under the PAER project (HUM2007-66479-C02-01/FILO).

References

  • Bartram, D. (2006). The internationalization of testing and new models of test delivery on the internet. International Journal of Testing, 6(2), 121-131.
  • Chalhoub-Deville, M., & Tarone, E. (1996). Assessment measures for specific contexts of language use, Paper presented at the Annual Meeting of the American Association for Applied Linguistics (18th, Chicago, IL, March 23-26, 1996).
  • Clarke, W. M. (1999). An assessment of foreign language training for english-speaking exporters. Journal of European Industrial Training, 23(1), 9-15.
  • Grosse, C. U. (2004). English business communication needs of mexican executives in a distance-learning class. Business Communication Quarterly, 67(1), 7-23.
  • Iwashita, N., & Grove, E. (2003). A comparison of analytic and holistic scales in the context of a specific-purpose speaking test. Prospect, 18(3), 25-35.
  • Maiworm, F., & Teichler, U. (1996). Study abroad and early career: Experiences of former ERASMUS students. higher education policy series 35. ERASMUS monograph no. 21
  • McCabe, P. J., Sheard, C., & Code, C. (2008). Communication impairment in the AIDS dementia complex (ADC): A case report. Journal of Communication Disorders, 41(3), 203-222.
  • Moreno,M., Regina Otero-Sabogal, R. & Newman, J. (2007) Assessing Dual-Role Staff-Interpreter Linguistic Competency in an Integrated Healthcare System, Journal of General Internal Medicine, 22(Suppl 2), 331-335.
  • O'Neill, T. R., Buckendahl, C. W., Plake, B. S., & Taylor, L. (2007). Recommending a nursing-specific passing standard for the IELTS examination. Language Assessment Quarterly, 4(4), 295-317.
  • Rouse, C. E., & Krueger, A. B. (2004). Putting computerized instruction to the test: A randomized evaluation of a ''scientifically based'' reading program. Economics of Education Review, 23(4), 323-338.

Author

Jesus Garcia Laborda, PhD, MA, MEd. has taught EFL for 15 years in Spain and the Balkans, and Spanish in the US. He lectures at the Gandia campus of the Polytechnic University of Valencia. He has published reviews and articles for many international journals both in Spanish and English, such as TESOL Quarterly, Language Teaching Research, Modern English Teacher and ESP World. His main interest at present is low stakes language testing.