We have developed a library for any application embedding haptic force feedback devices into Java GUI based interactive systems. It abstracts over the complications of inter-thread operation between graphical, haptic, and audio interfaces, providing a robust and reusable code library.
The DePIC research project has involved the design and implementation of cross-modal interactive systems for users living with visual impairments. The novelty of these systems is that they can be operated using differing sets of modalities, featuring both a conventional Graphical User Interface (GUI) for visual interaction and a more unusual audio-haptic interface.
Cross-modal audio haptic interfaces make use of sound and haptics to display the same information that is displayed by the visuals. The two interfaces are tightly integrated - changes to the system resulting from the input entered in one modality are rendered in real time in the other modality as well.
For the haptics we have been using force feedback devices such as the Geomagic® Touch™ and the Novint Falcon®. These allow users to literally feel a virtual scene by rendering it as a system of haptic forces such as resistance, friction, vibration etc. and can also be used as a source of input for example by pushing a button on the device or by triggering changes upon touching or moving virtual objects.
The haptic devices we have been using expose native programming API's (C/C++). However, a language such as Java is a very attractive choice for GUI programming as it allows faster prototyping and less error prone development than C/C++. Fortunately, the Java language also sports mechanism for incorporating native libraries into Java code, called Java Native Interface (JNI). Hence the decision in DePIC to use JNI to integrate the devices into Java GUI based programs.
After working on a number of prototypes in different application domains, some implementation patterns have started to emerge in the thread architecture of the applications. Specifically - the allocation of tasks among a number of threads and the way such threads communicate with one another can be reused from one application to the other, with only few minor changes. It’s worth mentioning that, on top of the integration between two different languages, the thread organization is not a minor and trivial aspect of the design of the application as one thread has to handle the haptic device - which runs at 1000 frames per seconds – in a loop, whereas another one has to provide constant reactiveness to the user input through the GUI.
We have captured these implementation patterns in the form a library of reusable code, leveraging on a practice that has proven to be sound and effective. The specific domain of the application as well as the specific device have been abstracted away. The library is therefore a useful tool for any application embedding haptic force feedback devices into Java GUI based interactive systems.
More detailed info on the library can be found in the manual available.