The processing of pen input in Windows is shrouded in mystery, and filled with messages and function calls only C programmers dare to evoke. Right? Well, sort of. While manipulating pen input is definitely something easy to do in the C language, you can do some interesting things in Visual Basic as well. Like improve recognition performance, for example. Let's go for a ride...

Recognition context
The results of the recognition process are returned to your program by the handwriting recognition engine, and when they are, the RCRESULT() event is fired by the pen control you are using- either the HEDIT (handwriting edit) or the BEDIT (boxed edit) controls that ship with VB. Both of these accept pen input, pass it to the Windows pen subsystem, and display the results. However, the RCRESULT() event is fired when the results of recognition are returned, and this is where things get interesting. In the RCRESULT() event, you gain access to the RCRESULT structure passed back as output from the recognition process. As it turns out, you can actually modify the recognition results based on "rules of thumb" such as "whenever the first character is a letter 'A' the second character is ALWAYS the letter 'Z' ", and so on. Think of the implications of this!

This small sample is a slightly simplified version of the pen sample that comes with VB3.0 Professional Edition. To run this example, you should have a computer running the pen extensions, and a full install of VB3.0/Pro. The sample code presented in this article allows you to examine the RCRESULT structure returned from the Windows Pen Extensions and gain an understanding of how to manipulate it. After you understand this example, you should examine the Guide To Pen Programming, found in the Windows Software Development Kit for more information on expanding this project or the sample code it is based upon. The sample upon which this discussion is based can be found in the VB/SAMPLES/PEN subdirectory.

Components
This sample uses some assets that come with VB3/Pro. First, there is the file PENAPI.TXT, which can be found in the VB/SAMPLES/PEN subdirectory. This file contains all the constants needed to manipulate the RCRESULT data structure from your VB program (see figure 1). Next, the project uses the PENCNTRL.VBX that can be found in your WINDOWS\SYSTEM directory after fully installing VB3/Pro (See the Project Window in figure 2.) One last thing- the sample here is running under Windows3.1, not Windows 95. While things have changed with Win95, the underlying concepts are the same, so you can use what's here, to good advantage, on either platform.

Theory
The pen system in Windows takes your raw pen input, bundles it up for processing, and sends it off to be recognized. The installed recognizer returns to your program an RCRESULT structure, which is a data structure containing, among other things, each character recognized.
Recognizers make a best guess at the characters entered by the user. One nice thing is that the guess is based on an array of POSSIBLE characters. The one with the highest confidence level is the one that gets displayed by the system. For example, the handprinted letter 'g' could be a '9' or some other character. The RCRESULT data structure, returned to your program by the recognizer, contains the best-guess as to recognition and an associated 'confidence factor'. All of this and more is contained in the RCRESULT. A pointer to the RCRESULT structure is actually PASSED BACK to your VB program via the RCRESULT event as a parameter.

Practice
The basic idea behind this example is that you can manipulate the output from recognition AFTER recognition is completed, but BEFORE the results are displayed. The hook that gives you this capability in VB is the RCRESULT event, which you will find attached to both the BEDIT and the HEDIT controls in PENCNTRL.VBX. It is with this hook that you can do some things reserved for C programmers- like peering inside the RCRESULT data structure. Why bother, you say? Because not only can you READ the RCRESULT, but you can write it, too. This means you can alter the recognition results on-the-fly under the RCRESULT event to improve the way your application works with pen input.

Programming
This example uses a BEDIT Control to receive input, and a Label control to display the RCRESULT structure returned by recognition. The confidence level for each recognized character is also displayed in the label. The running program looks like FIGURE 04.
To make this program work, you must first make a copy of the RCRESULT structure that is passed to your RCRESULT event. In truth, you do not get the RCRESULT structure- instead, you get a pointer, a C pointer to the structure itself. This would be pretty useless in VB, except that the PENCNTRL.VBX component contains a very special set of subroutines.

CPointerToVBType() takes a pointer to a structure and fills in a VB Type with the data. You, of course, are responsible for passing a VB Type that can accept all the data pointed to by the C pointer you specify. VBTypeToCPointer() does the opposite: it allows you to overwrite a data structure with new data contained in a VB Type. With these functions, you can pull off some very interesting VB tricks.

For a complete discussion of all the data types associated with the Windows Pen Extensions, please consult the Guide To Programming. In the example below we'll look at how to expose the characters in the RCRESULT structure. For a look at the nature of this data structure, see FIGURE 01, which shows the structure of this collection of data.

·Walking the code in the RCRESULT
event handler
When your app gets back the recognition results, the RCRESULT event fires, and the pointer the the RCRESULT data structure is passed in. The first thing we do is copy this data into the memory of our VB application, with a call to CPointerToVBType(), specifying where we want the data copied to. In this example, we fill our SAMPLERC VB data type with the information.

Sub bedSAMPLE_RcResult (RcResult As Long)
Dim SAMPLERC As RcResult 'RCRESULT Structure
Dim syeTable() As SYE 'Table of Symbol Elements
Dim iNumOfSymbols As Integer 'Num of Symbols
Dim i As Integer
Dim lpSYE As Long 'Pointer to symbol element
Dim lSYV As Long 'Symbol value
Dim SyvType As Long 'Symbol Type
'Get a copy of the RcResult Structure
CPointerToVBType ByVal RcResult, SAMPLERC, 80

Once we have that, we can spin through all of the characters in the 'symbol graph' portion of the RCRESULT data. Each RCRESULT structure contains a symbol graph, and it is the symbol graph that we are interested in. The symbol graph contains a pointer to the 'symbol elements', or individual items in the graph. It is these that we wish to display as we manipulate this application with the pen. The first thing we do is make a note of how many symbols are in the symbol graph. This is provided by the '.cSye' variable:

'Determine the number of symbols in the graph and get the address
iNumOfSymbols = SAMPLERC.SYGraph.cSye
This number will be used later to process all the symbols in the symbol graph. Are we having fun yet? Soon, we'll be displaying the recognition results. But first we have to copy all the symbol elements to an array we've predefined for processing. Again, we must make a call to CPointerToVBType():

lpSYE& = SAMPLERC.SYGraph.lpSYE
Rem - Allocate space for the graph and make a copy of it
ReDim syeTable(iNumOfSymbols)
CPointerToVBType ByVal lpSYE&, syeTable(1), iNumOfSymbols% * 12
Finally, we can look at each value the graph, by copying the value into a variable (lSYV, for 'long containing a Symbol Value') and then looking at the high-order part of this value to get the type of value it is. The type could be a gesture or a character:

'Clear the Graph label
Label1.Caption = " "
'Traverse the Symbol Graph
For i% = 1 To iNumOfSymbols%
lSYV& = syeTable(i%).Syv 'Get symbol value
SyvType& = lSYV& \ &H10000 'Calculate symbol type (SYVH_???)
'of symbol
'By looking at the HiWord
'Call subroutine to add symbol to graph
AddToGraph lSYV&, SyvType&, Format$(syeTable(i%).cl)
Next i%
End Sub
The AddToGraph subroutine simply takes the symbol value and type and confidence level (syeTable(i%).cl) and displays the recognition results with an associated confidence level. The results are displayed in the Label control Label1. The constants used in AddToGraph() are defined in PENAPI.TXT. See FIGURE 04 for a sample of how the program looks while running.

Summary
In this example, we have exploited the RCRESULT event procedure to display the results of recognition- on a character by character basis. We can read the RCRESULT structure into VB memory with CPointerToVBType(). And, while we have not done it here, we can update the data and then update the structure using the function VBTypeToCPointer(). Both of these routines are located in the PENCNTRL.VBX.

Reading the symbol graph found in the RCRESULT and modifying the contents is one more trick or the trade for pen developers using the Visual Basic programming system. Good luck with your own development. -
Sub AddToGraph (lSYV&, SyvType&, strCONFIDENCE As String)
Select Case SyvType&
Case SYVHI_ANSI 'If an ANSI Symbol, add it to the graph
Label1.Caption = Label1.Caption +
Chr$((lSYV& And &HFF)) + " <" + strCONFIDENCE + ">"
Case SYVHI_SPECIAL
'If the symbol is a Meta Symbol, replace it with a visible character
Select Case lSYV&
Case SYV_UNKNOWN
Label1.Caption = Label1.Caption + "? "
Case SYV_BEGINOR
Label1.Caption = Label1.Caption + "{ "
Case SYV_ENDOR
Label1.Caption = Label1.Caption + "} "
Case SYV_OR
Label1.Caption = Label1.Caption + "| "
End Select
Case SYVHI_GESTURE 'If the symbol is a gesture, print the gesture name
Select Case lSYV&
Case SYV_CLEAR
Label1.Caption = " - CLEAR Gesture"
Case SYV_EXTENDSELECT
Label1.Caption = " - EXTEND SELECT Gesture"
Case SYV_UNDO
Label1.Caption = " - UNDO Gesture"
Case SYV_COPY
Label1.Caption = " - COPY Gesture"
Case SYV_PASTE
Label1.Caption = " - PASTE Gesture"
Case SYV_CLEARWORD
Label1.Caption = " - CLEAR WORD Gesture"
Case SYV_USER
Label1.Caption = " - USER Gesture"
Case SYV_BACKSPACE
Label1.Caption = " - BACKSPACE Gesture"
Case SYV_TAB
Label1.Caption = " - TAB Gesture"
Case SYV_RETURN
Label1.Caption = " - RETURN Gesture"
Case SYV_SPACE
Label1.Caption = " - SPACE Gesture"
Case SYV_CIRCLEUPA To SYV_CIRCLEUPZ
Label1.Caption = " - Circle " +
Chr$(lSYV& - SYV_CIRCLEUPA + Asc("A")) + " Gesture"
Case SYV_CIRCLELOA To SYV_CIRCLELOZ
Label1.Caption = " - Circle " +
Chr$(lSYV& - SYV_CIRCLELOA + Asc("a")) + " Gesture"
End Select
End Select
End Sub

- Daniel Mezick

Daniel Mezick is founder and president of New Technology Solutions, a consulting and training firm based in New Haven, CT. He began pen computing with the PenPoint 1.0 beta. His company now develops pen applications for Fortune 500 firms, using a variety of development tools. You can reach him at (203) 239-6874.