Apple's Human Interface Research
Written by David Tebbutt, MacUser 10/91 item 01 - scanned
Apple's researchers are devising innovative interfaces to keep the company at the forefront of computer/human interaction.
Michael Chen was the person mainly responsible for the pen and finger manipulation interface for the hand-held computers previously described in MacUser (Inside Apple, Vol 7 No 19, 20 September 1991, p68). He is an expert on input devices and how to make effective use of them at the screen interface. He is also the inventor of a brand new concept in 3D manipulation called the `virtual sphere'. This second article on the future of the Mac looks at this sphere and at Chen's involvement in the Leonardo project, which deals with new ways of achieving 3D design and 3D manipulation.
Another member of the project, Ian Small, is working on the use of interface animation to make computers more helpful to their users. His work looks at the possibilities of animating applications and functions within applications. Although the two men don't work closely together, they are united by a common background at the University of Toronto. Their work has been supported by Apple Computer through grants and its graduate student internship programme. Chen is now a full-time Apple employee, while Small is splitting his time between working at Apple and completing his PhD at Toronto.
The Virtual Sphere
Chen's journey of discovery was stimulated by Bill Buxton, a leading thinker in the human interface world. In 1985, when studying under Buxton at Toronto, Chen asked him why 3D interaction was so poor. Instead of answering, Buxton challenged Chen to do something about it. At the time, Buxton suggested working on an interface for performing 3D rotation by dividing a touch pad into nine squares (three by three) so that vertical and horizontal movements could be interpreted on the x and y axes, while circular movements, which miss the central square, could take care of the z axis.
Chen produced a working system based on Buxton's idea and then started thinking about an even more natural approach, that of continuous rotation in all three dimensions using just a 2D device such as a mouse as the controller. He invented a technique of encasing the object to be manipulated within a transparent sphere which can be `rolled' using the input device. A user-visible reference circle surrounds the object and a pointer associated with the input controller is moved over the surface, or around the outside, of the invisible sphere.
In the case of the mouse, holding down the button causes the sphere and its contents to rotate according to the cursor movement. The user gets the illusion that he is manipulating the content directly. Any cursor movement beyond the circle is treated as if it were on the edge.
User studies conducted by other members of the Human Interface Group (HIG), part of Apple's Advanced Technology Group, tested the sphere for speed and accuracy against four other rotational methods. The results showed that the sphere was faster for complex rotations and just as effective as more conventional sliders for movement around a single axis.
Watching The Architect
The HIG is very good at studying the way people work, and not just how they work with computers. As part of the group, the Leonardo project spent a lot of time observing an architect during the initial design process, as his ideas took shape on successive layers of tracing paper. It wasn't until his concepts were nearly finalised that he then turned to the computer (AutoCAD on the PC) to draft the specification for the contractors.
Yin Yin Wong, a visual designer and researcher at the HIG, noted that the computer doesn't support a continuum from initial rough sketches to finished design - it demands concrete information before the user is ready. Early in the design phase, the architect wants to express a general idea without committing to a single solution. For example, the architect draws a single line to represent a division of space. The value and weight of the line is used to express the nature of the divider. So a thin line with a lighter value might be variously interpreted as a screen, a pane of glass or a waterfall.
The computer does have advantages for the architect. It can help visualise things like the effect of different positions of the sun. It can confirm ideas and allow presentation of the design to clients in the form of rendered models.
The HIG has produced a demonstration of how the initial rough stages of architectural design might be accomplished on the Mac by giving the user the equivalent of sheets of tracing paper. In the design of a house, for example, the first layer might contain the land boundaries, contours and site of trees. Another piece can then be placed on top, causing the first to show through faintly - just as it would with tracing paper. This contrasts with traditional CAD systems which use layers but make them all of the same intensity when displayed together.
A rough shape of the living areas might be sketched on. A room with a view, an accessible garage, a sun room and so on might be drawn as rough circular blobs. As the idea takes shape, the architect can place a further tracing layer on which the first firm outlines begin to be seen. Once again, the lower levels show through, but more and more faintly. The layers can be moved easily and any layer can have extra material drawn on it by bringing it to the top.
Such a system would give the architect a historical trace of the design development as he or she builds it up from first principles.
The Leonardo Project
The team realised that the use of amorphous objects in the computer world is difficult to do in 2D and even harder in 3D, because the computer is too exact. Parallel research is now being conducted into the process of general 3D design on the one hand and the development of 3D manipulation demonstrations on the other.
The model chosen for the 3D manipulation demonstrations was the placement of furniture in a room. This involves sliding, rotating and lifting actions whenever the cursor is on an object. A summer intern at the HIG, Stephanie Houde, introduced the concept of '3D handles'. When an object is selected, a box is drawn around it with handles on the top and sides, to indicate the ways in which the object can be moved.
A thumbwheel mouse was introduced. This, as the name suggests, has a milled wheel attached to its side which can be turned with the thumb. The intention was to give the user an extra degree of freedom when manipulating objects in three dimensions. A chair, for example, can be moved across the floor in any direction by using the mouse and rotated on the spot using the thumbwheel.
Laurie Vertelney and Penny Bauersfeld designed a living room which introduced the idea of having people in the scene to represent different points of view and to give an idea of scale. They might be standing by a door or sitting in a chair. The user is able to get a different view of the furniture placement by electing to `be' one of these people. The thumbwheel on the mouse can swivel the head of the 'person'. The mouse itself can change the person's position, making it appear that the user is walking across the floor. The prototype even included footstep sounds.
The thumbwheel mouse raises the issue of 'handedness'. The wheel has to be on the left or right side of the mouse, depending on the user's preference. A further development, that of a roller mouse, puts a roller across the front top edge of the mouse where it can be operated by the forefinger of either hand.
This project is still in progress. The team has to address many issues, including how to select and manipulate multiple objects and what sorts of tasks are most suitable for these 3D manipulation techniques.
Animation At The Interface
The Mac interface as it stands is not always as helpful as it could be. What, for example, is a new user to make of an icon that looks like a kidney bean? The answer, as we all know, is a freehand drawing tool. Ian Small has been working on ways of using animation to make information at the user interface clearer. Animated icons for programs, for example, can help. The HIG has demonstrations of a 'pong' icon in which a tennis ball is bouncing around and an icon for a finger paint program which shows a finger painting something.
Users have responded well to animated tool items. Small and his colleagues took the HyperCard toolbox and, ignoring the top row (browse button and field) implemented animations of the other 15 tools. Each animations was kept within the boundary of the tool icon, so it had a 22 by 20 pixel area. Eric Martin, an experienced animator familiar with the Mac, was invited to create the prototype animations, each one involving 400 frames. The researchers gained some unexpected insights when they tried the animations out on users. One user, for example, described the paint bucket as a graduation cap (mortar board). Even after its real identity had been revealed, the user still referred to it as ``the hat that pours paint''. Two users thought the kidney bean icon was only for drawing kidney beans. If anything, these perceptions reinforced the need for animated tool icons.
Mistakes were made with the initial animated icons. The icon for the eraser wiped a black area clean to reveal an eye. It was done this way to emphasise HyperCard's foreground/background properties. This simply didn't make sense to novice users so the eraser now leaves plain white. The researchers also found that the drawing icon animations could be improved with the addition of a cursor and clicking sounds to signify mouse button actions.
When the users were allowed to practise on the tools, they weren't told about the animation. They discovered it and, according to Small, were delighted. They still made a few wrong assumptions, though. Because the spray can created a brick wall pattern, this became known as the brick wall tool. The paint can still caused a problem for one user when asked to draw a house, the user turned to the paint can tool to get a chimney shape. This mistake turned the drawing completely black. If you look at the paint tool icon and the house drawing, you will see how the confusion arose.
The icon animations come to life when the cursor is placed over a tool. First, the icon dissolves, then the animation begins. Unfortunately, one or two users thought the dissolving icon meant the tool couldn't be used. The researchers considered having all the tools animated all the time, but rejected the idea because the screen would be too busy.
The important thing for the HIG is that animations work. Now it will try and identify the rules which underpin successful animation at the interface.
MacUser would like to thank Michael Chen, Yin Yin Wong, Ian Small, Laurie Vertelney and S Joy Mountford for their help in researching this article.