Equation tutor for sighted and visually impaired children

AAFG 2005 Math-Puzzle: Equation Tutor for Sighted and Visually-Impaired Children Jarno Jokinen Department of Computer Sciences University of Tampere Finland jarno.jokinen@uta.fi April, 2005 Math-Puzzle introduction • communication of mathematics is usually visual • formulas, diagrams, graphs etc • it is very difficult for blind and partially sighted people / students to mathematics and is one of the biggest obstacles for them in school and at the university Math-Puzzle most of the work reported in developing techniques that deal with mathematics can be presented through next categories: • tactile as in Braille and other raised representations • haptic or forced feedback devices represent shapes and curves • tonal representing equations and graphs (sonification / audification) • audio aids that read equations with tools to help in reading process • integrated /multimodal approaches Math-Puzzle Introduction into the games http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm Math-Puzzle http://www.gamealbum.com/keyword/math/ Math-Puzzle project impetus • created a game instead of making tests with an existing game • mathematical game for blind children • after different ideas, Math-Puzzle was chosen • training in logics, memory and equations’ manipulation strategy Math-Puzzle the problems in question • is it possible to solve equations using only short speech cues? • what are the limitations? • what is the easy way for blind interaction to edit the equations in static or dynamical puzzle? (memory capacity or external memory aid see next slide) • how long does it take to solve a puzzle? • what are the parameters of the gameplay progress and player performance? • how much does it make difference when player has visual feedback or all the tasks (including navigation within the game-field) are presented through sounds? • how and in which order the equations could be solved? (strategies) Math-Puzzle different matrix approaches • static puzzle on the left • dynamic puzzle on the right Math-Puzzle game concept • 5x5 matrix with one equation in each row • equations are predefined and randomly picked for the matrix • the goal of the game is to change the places of the equation members so that all of the equations are true • the figures (only in the same column) can be swapped by clicking one number and then the other one • operators cannot be swapped as well other controls are: • alt+R -> new game (reset) • alt+M -> minimize / maximize browser window • alt+S -> show / hide figures and operators • the puzzle completion time and the number of moves are calculated • the game is currently implemented only in www with limitations Math-Puzzle testing procedure • technically aware adults (age ranged from 26 to 34) • 10 games per player with all three playing modes: 1st visual, 2nd blind (hidden labels with sound cues), 3rd blindfolded • laptop pc with external mouse and headphones Math-Puzzle problems • sound feedback is not supported with mozilla browsers • completing the fourth equation usually completes also the fifth • if not, the answers are crossing each other in a way that makes it very difficult to solve the puzzle • the program sometimes gives or more correct equations at the start which may also result in an error on the browser Math-Puzzle results and discussions • the game (blind mode) is hard for adults that can see • what about kids that can’t? They have better spatial understanding • some matrixes can be a lot faster solved than others – repetitions needed for a good estimate of skills • if you wan’t to hear the sound cues you need to be patient • sound is heard when the mouse is moved over the square, so if you want to hear it again you must move the mouse away and over again • completing the fourth equation is crucial as explained on next slide Math-Puzzle about the fourth equation OK OK OK OK Math-Puzzle conclusions • game requires a lot of memorizing, but there are strategies that help • completing equations in some order • using mathematical rules (division and multiplication) • when playing the game blindfolded, the increase in speed was bigger than when playing the visual version Math-Puzzle References http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm http://www.gamealbum.com/keyword/math/ Children’s math project http://www.udel.edu/educ/cmp2/ http://www.educational-software-directory.net/math/ Lambda-project: Linear Access to Mathematic for Braille Device and Audio-synthesis http://www.lambdaproject.org/ Karshmer, A.I., Gupta, G., Gillan, D Architecting an Auditory Browser for Navigating Mathematical Expressions, ICCHP 2002, LNCS 2398, p 477 http://link.springer.de/link/service/series/0558/papers/2398/23980477.pdf Gaura, P REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002, LNCS 2398, p 486, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdf Fitzpatrick D Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical Material, ICCHP 2002, LNCS 2398, p 494, http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf http://www.computing.dcu.ie/~dfitzpat/publications.html Math project, http://www.cs.york.ac.uk/maths/index.html Prosody in Mathtalk http://www.cs.york.ac.uk/maths/robert/prosody.html Mathematical Access for Technology and Science, http://www.papenmeier.de/reha/research/mathe.htm Edwards, A D N., Stevens, R D and Pitt, I J Représentation non visuelle des mathématiques, (translated by A Assimacopoulos) in A B Safran and A Assimacopoulos (editors) Le Déficit Visuel, Éditions Masson, pp 169–178 (1995), http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.ps Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999