name: JSGFToken

synopsis:

g++ [flags ...] file ... -l /isip/tools/lib/$ISIP_BINARY/lib_mmedia.a

#include <JSGFToken.h>

JSGFToken();
boolean setToken(const String& arg_a);
boolean printToken();
String getTerminal();
boolean setCoordinate(long row_a, long col_a);

quick start:

JSGFToken jt;
String token_str(L"public");
jt.setToken(token_str);
jt.printToken();

description:

The JSGFToken class is used to store and manage various types of JSGF grammar tokens tokenized from given JSGF grammar strings. For a good reference on JSGF grammar, see:

Java Speech Grammar Format Specification,

http://java.sun.com/products/java-media/speech/forDevelopers/JSGF/index.html

This class is mainly used by JSGFParser class to convert JSGF grammar to ISIP DiGraph representation.

dependencies:

public constants:

define the class name:

static const String CLASS_NAME = L"JSGFToken";

define i/o related constants:

static const long DEF_TOKEN_TYPE = 0;

static const String DEF_HEADER = L"";

static const String DEF_KEYWORD = L"";

static const String DEF_GRAMMAR_NAME = L"";

static const String DEF_IMPORT_GRAMMAR = L"";

static const String DEF_RULENAME = L"";

static const String DEF_TERMINAL = L"";

static const String DEF_OPERATOR = L"";

static const String DEF_TAG = L"";

static const String DEF_COMMENT = L"";

static const Float DEF_WEIGHT = 0;

static const long DEF_TERM_ROW = 0;

static const long DEF_TERM_COL = 0;

error codes:

error code indicating JSGFToken class general error:
```
static const long ERR = 50200;
```

public data:

token type:

Long token_type_d;

grammar header:

String header_d;

keyword:

String keyword_d;

grammar name:

String grammar_name_d;

import grammar name:

String import_grammar_d;

rulename:

String rulename_d;

terminal symbol:

String terminal_d;

operators:

String operator_d[2];

tag:

String tag_d;

comment:

String comment_d;

weight:

Float weight_d;

location of terminal in graph:

long term_row_d;

long term_col_d;

static memory manager:
```
static MemoryManager mgr_d;
```

required public methods:

static methods:

static const String& name();

static boolean diagnose(Integral::DEBUG level_a);

debug methods:

boolean debug(const unichar* msg_a) const;

destructor/constructor(s):
```
~JSGFToken();
```
```
JSGFToken();
```
assign methods:
```
boolean assign(const JSGFToken& arg_a);
```

operator= methods:

JSGFToken& operator=(const JSGFToken& arg_a);

equality methods:

boolean eq(const JSGFToken& arg_a) const;

operator== methods:

boolean operator==(const JSGFToken& arg_a) const;

i/o methods:

boolean read(Sof& sof, long tag, const String& cname = CLASS_NAME);

boolean write(Sof& sof, long tag, const String& cname = CLASS_NAME) const;

memory management methods:

static void* operator new(size_t size);

static void* operator new[](size_t size);

static void operator delete(void* ptr);

static void operator delete[](void* ptr);

static boolean setGrowSize(long grow_size);

boolean clear(Integral::CMODE ctype_a);

class-specific public methods:

set methods:

boolean setToken(const String& arg_a);

boolean setCoordinate(long row_a, long col_a);

get methods:
```
String getTerminal();
```
```
long precedence();
```
print methods:
```
boolean printToken();
```

private methods:

methods to identify different types of tokens:

boolean isHeader(const String& arg_a);

boolean isKeyword(const String& arg_a);

boolean isGrammarName(const String& arg_a);

boolean isImportGrammar(const String& arg_a);

boolean isRulename(const String& arg_a);

boolean isTerminal(const String& arg_a);

boolean isOperator(const String& arg_a);

boolean isTag(const String& arg_a);

boolean isComment(const String& arg_a);

boolean isWeight(const String& arg_a);

examples:

This example shows how to use JSGFToken class to set and manipulate tokens based on input string unit of JSGF grammar.

// isip include files
//
#include <JSGFToken.h>
       
// declare two JSGFToken instances
//
JSGFToken jt_1, jt_2;

// define two JSGF token strings
//
String token_str_1(L"<polite>");
String token_str_2(L"hello");
       
// set token
//
jt_1.setToken(token_str_1);
jt_2.setToken(token_str_2);

// print the first token
//
Console::put(L"The first token is:");
jt_1.printToken();

// get and output the terminal symbol of the second token 
//
String symbol = jt_2.getTerminal();
Console::put(L"\nThe terminal symbol of the second token is:");
Console::put(symbol);
       
// set the second token's location in graph
//
long row = 3;
long col = 4;

jt_2.setCoordinate(row, col);

// output the token information after the settings
//
Console::put(L"\nThe information about the second token is:");
jt_2.debug(L"token");

notes:

Unlike most of the ISIP foundation classes, JSGFToken, mainly used to support JSGFParser class, defines all its member data as public so that JSGFParser can manipulate tokens more straightforwardly. JSGFToken is not used independently as a general class.