name: LanguageModel

synopsis:

g++ [flags ...] file ... -l /isip/tools/lib/$ISIP_BINARY/lib_pr.a

#include <LanguageModel.h>

LanguageModel();
HierarchicalDigraph& getHDigraph();
boolean setHDigraph(const HierarchicalDigraph& hg);
boolean convert(LanguageModel& lm, ALGORITHM format);
boolean setFormat(ALGORITHM format);

quick start:

LanguageModel lm;
Sof lm_file;
lm_file_in.open(L"lm_file_in.sof", File::READ_ONLY, File::TEXT);
lm.read(lm_file_in);
lm_file_out.open(L"lm_file_out.sof", File::WRITE_ONLY, File::TEXT);
lm.write(lm_file_out);

description:

In addition to the acoustic information, linguistic information is extremely important in recognizing and understanding natural speech. For a language, there are a number of components that are crucial to achieving the goal of spoken language understanding system. These components include the syntax and semantics of the language (the rules that determine what sequences of words are grammatically well-formed and meaningful), the lexical knowledge of the language (vocabulary definition and word pronunciation) and the knowledge of the pragmatics of language (the structure of extended discourse, and what people are likely to say in particular contexts). Two good references to the introduction on Formal Language Theory are:

Spoken Language Processing - A Guide to Theory, Algorithm, and System Development

Statistical Methods for Speech Recognition

The two fundamental components of the Formal Language model are: the grammar and the parsing algorithm. The grammar is the set of permissible rules for the structure in a language that is represented in either a graph or text format. The parsing algorithm is a technique to analyze the sentence to make sure that the sentence under consideration follows the rules laid down by the grammar. The LanguageModel Class corresponds to the first component, the grammar.

The LanguageModel class describes the various components of the grammar. These components are constraints due to syntax and semantics (word-level grammar). These word-level constraints can be expanded to phone-level using the lexical information of the language. The phones are often defined by models based on the pragmatics of the language (context). For speech, HMM's are the most popular way to model the phonetic models. Each state of these HMM's are modeled by a mixture of Statistical models. For this case, the LanguageModel class sets up a search space comprising of three search levels - word level, phone level and state level.

Currently we support three formats to represent the grammars. The first one is our internal format - DiGraph that we employ for speech training as well as decoding. The second and third, JSGF (Java Speech Grammar Format) and XML-SRGS (XML Speech Recognition Grammar Specification), are platform-independent textual representations. For additional information on these format, see:

Java Speech Grammar Format Specification

http://java.sun.com/products/java-media/speech/forDevelopers/JSGF/index.html

XML Speech Recognition Grammar Specification

http://www.w3.org/TR/speech-grammar

LanguageModel class can also be used to convert JSGF and XML grammars to the ISIP Digraph format and vice-versa. The JSGFParser and XMLParser classes are used to read the grammars in the JSGF or XML formats, respectively.

dependencies:

public constants:

define the class name:
```
static const String CLASS_NAME;
```

define the algorithm choices:

 enum ALGORITHM { UNKNOWN = 0, IHD, JSGF, XML, BNF, ABNF, DEF_ALGO = UNKNOWN };

define the implementation choices

enum IMPLEMENTATION { HD, DEF_IMPL = HD };

define the static NameMap objects:

static const NameMap ALGO_MAP;

static const NameMap IMPL_MAP;

define i/o related constants:

static const String PARAM_ALGO;

static const String PARAM_IMPL;

static const String DEF_PARAM;

static const long DEF_TAG;

default values

static LanguageModelUndefined NO_LANGUAGE_MODEL;

error codes:

error code indicating LanguageModel class general error:
```
static const long ERR = 100102;
```

protected data:

the virtual LanguageModel object

LanguageModelBase* virtual_lm_d;

format:

ALGORITHM format_d;

implementation:

IMPLEMENTATION implementation_d;

declare the memory manager:
```
static MemoryManager mgr_d;
```

required public methods:

static methods:

static const String& name();

static boolean diagnose(Integral::DEBUG debug_level);

debug methods:

boolean debug(const unichar* msg_a) const;

boolean setDebug(Integral::DEBUG debug_level);

destructor/constructor(s):
```
~LanguageModel();
```
```
LanguageModel();
```

assign methods:

boolean assign(const LanguageModel& arg_a);

equality methods:

boolean eq(const LanguageModel& arg_a) const;

i/o methods:

long sofSize() const;

boolean read(Sof& sof_a, const String& tag_a, SearchLevel& level_a);

boolean write(Sof& sof_a, long tag_a, const String& name_a) const;

readData(Sof& sof, const String& pname = DEF_PARAM, long size = SofParser::FULL_OBJECT, boolean param = true, boolean nested = false);

boolean writeData(Sof& sof, const String& pname = DEF_PARAM) const;

memory management methods:

 boolean clear(Integral::CMODE cmode = Integral::DEF_CMODE);

class-specific public methods:

declare class-specific public methods:

const String& className() const;

RuleModel getRuleModel();>

boolean setRuleModel(const RuleModel& hg);

ohter methods:

declare other methods:

boolean convert(ALGORITHM format)>;

boolean assign(const LanguageModelBase& arg);

boolean eq(const LanguageModelBase& arg) const;

boolean setFormat(ALGORITHM format);

ALGORITHM getFormat();

boolean minimizeGraph();