Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id iBEKqhU02989; Tue, 14 Dec 2004 15:52:43 -0500 (EST) Date: Tue, 14 Dec 2004 15:52:43 -0500 (EST) Message-Id: <014901c4e21e$7c44dac0$203b7780@april> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: "April L. Zenisky" <azenisky@educ.umass.edu> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:817] Re: guidance in developing job-related X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-transfer-encoding: 7BIT Content-type: text/plain; reply-type=original; charset=iso-8859-1; format=flowed X-Mailer: Microsoft Outlook Express 6.00.2900.2180 Status: O Content-Length: 8282 Lines: 217 Hi Patti, Thanks for your thoughts! Accessibility is definitely one aspect that computerized testing can help with, particularly from the perspective of the principles of Universal Test Design. Indeed, a computerized test _can be_ preferable to paper for exactly all the reasons you mention -- there's a huge potential there for the test medium to be customized by and for the individual. One thing that I wonder about in this regard is computer familiarity and different learners' knowledge of the buttons and bells and whistles to make some of those adjustments? Tutorials are by and away critical to any computerized testing application, as are opportunities for navigating around practice systems prior to the "for real" test administration. The issue of computer familiarity is one that is very important to us in our work in Massachusetts. Given the wide ranges of 1)student academic proficiencies and 2) students' computer familiarity levels and 3) computer use in programs, the interaction between a test-taker and a computer to measure academic skills is an issue we are focusing on -- to what extent would content and not computer familiarity (construct-irrelevant variance) be tested by a computer-based test in our target population? --april Just a footnote: here's a link to read some more about Universal Test Design, for those whose interest may be piqued.... [Thurlow, M., Quenemoen, R., Thompson, S., & Lehr, C. (2001). Principles and characteristics of inclusive assessment and accountability systems (Synthesis Report 40). Minneapolis, MN: National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/Synthesis40.html] ----- Original Message ----- From: "Patti White" <prwhite@MadisonCounty.NET> To: "Multiple recipients of list" <nifl-assessment@literacy.nifl.gov> Sent: Tuesday, December 14, 2004 2:32 PM Subject: [NIFL-ASSESSMENT:816] Re: guidance in developing job-related > April, > > I'm glad to hear you're working on developing more computerized tests. My > interest is focused on accessibility for people with disabilities, and a > computerized assessment is inherently far more accessible for many types of > disabilities, especially when combined with other accessible > features/software that might interface with the assessment, like JAWS or > other text-to-speech software programs. It would help, too, if the student > were able to adjust the size/type of font, screen colors, and be able to > highlight text for visual tracking. The presentation of one question at a > time is also helpful, as well as the use of a keyboard or speech-to-text > software for short-answer or essay questions. I'm sure there's many more > possibilities.... > > Thanks for your good work! > Patti > > Patti White > Disabilities Project Manager > Arkansas Adult Learning Resource Center > prwhite@madisoncounty.net > > ----- Original Message ----- > From: April L. Zenisky > To: Multiple recipients of list > Sent: Tuesday, December 14, 2004 1:04 PM > Subject: [NIFL-ASSESSMENT:815] Re: guidance in developing job-related > > > > Hi, all, > To follow up with Marie's reposting, this was a question that was especially > interesting to me as my > own research interests as a test developer involve alternate item types (not > multiple-choice) that > are computer based. Specifically, I am interested in uses of innovative or > novel types that are > highly engaging to the test-taker but still auto-scored. In many cases, > there might be one correct > answer, but the questions might be written to allow for more than one right > (or wrong) answer. > > For example, the different item formats for use in computerized testing > allow you to incoporate > different actions such as clicking, drag-and-drop, scrolling, and typing. > The drag-and-drop option > is nice because people can rearrange onscreen items in different ways given > the frame of the > question (for example, put things in some order (numerical, alphabetical, > chronological, etc.)) and > it lets them visualize the groupings they are creating. > > So, here in Massachusetts my colleagues at the Center for Educational > Assessment and I are working > with the Massachusetts DoE and practitioners in the state to develop new, > computerized assessments > that are directly based on the Massachusetts curriculum. The MA assessments > will be different from > the computerized BEST Plus in that the student willl enter answers directly > into the computer, which > is different from the BEST Plus' approach (which works well for that domain > being tested -- a test > administrator scores a student's spoken answers on a few dimensions and > enters those into the > computer). > > One thing I'm interested in learning more about myself about is people's > experiences with adult > learners and computerized testing. Some work I'm involved in now is finding > out about adult > learners' familiarity with computerized response actions (clicking, > drag-and-drop, scrolling, and > typing). Does anyone out there have any experience with or thoughts about > using computer technology > for testing with adults that they'd be interested in sharing with the list? > More broadly, for > practitioners, what do you find that your students can do with respect to > using a computer and what > skills do you see them needing assistance with? What might you consider to > be the important > navigational and other features of computerized activities/assessments? > > I'm looking forward to hearing from you! > --april > > > > *********************************************** > April L. Zenisky, Ed.D. > Director of Computer-Based Testing Initiatives > Center for Educational Assessment > University of Massachusetts Amherst > Amherst, MA 01003 > 413.577.2180 > azenisky@educ.umass.edu > > > > > > > > > > ----- Original Message ----- > From: "Marie Cora" <marie.cora@hotspurpartners.com> > To: "Multiple recipients of list" <nifl-assessment@literacy.nifl.gov> > Sent: Tuesday, December 14, 2004 12:50 PM > Subject: [NIFL-ASSESSMENT:814] guidance in developing job-related > assessments for critical thinking > > >> Hi everyone, >> >> I am re-posting Julie's question from October 14 because it is relevant >> to the discussion this week, and I don't think she ever received much in >> reply. Julie - have you made progress on this project to date? Perhaps >> April, and others who wish to, can comment on some of your questions. >> marie >> >> >> >> I am working with large organization to develop assessments for managers >> in >> training. They want the questions to focus on measuring whether or not >> a >> management trainee can think critically and problem solve. The questions >> will focus >> on IMPACT and INDICATIONS of daily company procedures. >> >> The types of questions need to assess if the trainee understands the >> impact >> of a particular procedure on the productivity, bottom-line, etc. for the >> >> business. We also want to assess whether or not the trainees can >> identify the >> indicators that something is, or is not, going (or set up) according to >> procedures. >> >> For example, does the trainee understand the IMPACT if department >> materials >> are unorganized and/or dirty? That the mailbag has gone unopened? Can >> s/he >> identify the INDICATORS that something is not running according to >> procedure, is >> out of place or that an employee is not performing their job >> requirements? >> >> This assessment needs to written in standardized format (i.e., multiple >> choice, true false, some fill in the blank and a few short answers). >> >> I've created many assessments for basic skills, so have an understanding >> of >> assessment development. However, I can see that testing intangible >> problem >> solving skills, will require a different approach. (By the way, the >> precise >> training modules have not been developed.) >> >> I am working with the company to identify the proficiencies, but would >> very >> much appreciate samples of how to phrase the questions so that they will >> >> accurately measure the proficiencies in the above mentioned format. >> >> Does anyone have samples or guidelines I might use? >> >> Thanks for your input. >> >> Julie Baker >> >> >> >> >> > >
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:26 EST