JPH04182000A - Continuous speech recognizing device - Google Patents

Abstract

PURPOSE:To drive a phoneme context dependence type phoneme model by driving an HMM(Hidden Markov Model) phoneme matching part which matches a phoneme sentence by using a phoneme context dependent HMM phoneme model matching a predicted phoneme context and finding the presence probability of the predicted phoneme. CONSTITUTION:A phoneme context prediction part 107 predicts the phoneme context of a phoneme predicted by a predictive LR (Left to Right) parser part 108, the phoneme context dependence type phoneme model 102 matching the environment of the phoneme context is driven, and the presence probability of the phoneme predicted is driven, and the presence probability of the phoneme predicted by the phoneme context prediction part 107 is found by driving the HMM phoneme matching part 101. Namely, a phoneme in input speech data is predicted by using an LR table 106, the phoneme context prediction part 107 predicts the phoneme context of the periphery of the predicted phoneme by using the LR table 106, and the prediction result is verified by the phoneme matching function of an HMM phoneme recognition part. Consequently, the phoneme context dependency type HMM phoneme model 102 corresponding to an LR parser phoneme context can be driven.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a continuous speech recognition device, and in particular, it is used to predict LR child table input speech data, and this prediction is verified by the phoneme matching function of the HMM phoneme recognition device. The present invention relates to a continuous speech recognition device that performs speech recognition and language processing in a unified manner.

[Prior Art] Conventionally, there is an HMM-LR method as a continuous speech recognition method that performs speech recognition and language processing in a unified and efficient manner. With this method, speech recognition and language processing, which had previously been performed separately, are handled in a unified manner, allowing highly reliable and efficient processing to be performed without the need for intermediate data such as bunsetsu lattices. It can be done.

In the HMM-LR method, the existence probability of a phoneme predicted by a language processing method called the LR method (LR parser) is
It is calculated using a speech recognition method called the (Hidden Markov Model) method. Below, HMM-LR
Before explaining the method, the LR method and HMM method will be explained.

In the field of computational geometry, particularly in the field of programming language processing systems, sufficient research has been conducted on syntactic analysis techniques, one of which is a method called the LR parser. L
The R parser is a type of so-called 5HIFT-REDUCE parser, and analyzes input symbols while reading them from left to right. The LR parser internally maintains something called a state, and uses the current state and input symbols to determine the next action to take.The operations of the LR parser include (1) ACCEPT (2 ) ERROR (3) SHI FT (4) REDUCE are allowed. ACCEPT indicates that the input string to the LR parser has been accepted. ERROR indicates that the input symbol string to the LR parser has been accepted.
Indicates that the input string to the parser was not accepted.

SHI FT pushes the input symbols currently seen by the LR parser and the current state onto the stack. Translate the symbol at the top of the REDUCE model into a larger unit. R
At EDUCE, state symbols and input symbols are removed from the stack by the number of grammar rules on the right side of the grammar rule used.

To determine the behavior of the LR parser based on the current state and input symbols, a table called the LR child table is referred to. L
The R child table must be prepared in advance before being analyzed by the LR parser. It can be constructed mechanically from the LR child table and grammar rules.

Figure 4 is a diagram showing an example of grammar rules, and Figure 5 is a diagram showing an example of grammar rules.
It is a figure which shows the example which converted the LR child table of the grammar rule shown in the figure.

As shown in FIG. 5, the LR child table consists of two tables called the ACTION table and the GOTo table. The ACTION table is a table in which the state of the LR parser is described in the vertical axis direction and input symbols are arranged in the horizontal axis direction, and one section of the table describes the action that the LR parser should take. In FIG. 5, the operation described as ace means ACCEPT, and the space in the table represents ERROR. The symbol starting with S represents 5HIFT, and the number written after S is the state that the LR child table should be in after performing the 5HIFT operation.

The symbol starting with r represents REDUCE, and the number n written after r indicates that a reduction operation using the nth grammar rule is performed.

After the LR parser performs the REDUCE operation, the GOT
Refer to O table. The GOTO table is a table that describes the state of the LR parser along the vertical axis and describes non-terminal symbols along the horizontal axis. The LR parser uses the nonterminal symbol created as a result of the REDUCE operation and the current state to
The new state is determined by the TO table. The state of the LR parser at the time parsing starts is 0, and the LR parser performs an ACCEPT operation and accepts the input symbol string, or
The analysis ends when the ERROR operation is performed and the input symbol string is not accepted.

On the other hand, in the field of speech recognition, there is a method of recognizing utterances by regarding them as probabilistic state transitions, which is called the HMM method.

FIG. 6 is a diagram of a typical phoneme model used in the HMM method. The method of phoneme recognition using HMM will be described below with reference to FIG. Each string of the HMM is given the probability of transition between states and the output probability of the symbol.
A symbol string is output stochastically based on these values. H.M.
In order to perform phoneme recognition using the M method, HMMs for each type of phoneme are prepared in advance, and the probabilities of the phoneme HMMs are learned so that each symbol string of learning phoneme data is output with the highest probability. , Next, for the unknown speech data symbol string, the probability that the symbol string is output from all HMMs is calculated, and the phoneme corresponding to the HMM with the highest probability is taken as the recognition result.

The operation of calculating the probability for this unknown speech data is called phoneme matching. This operation is realized, for example, for the HMM shown in FIG. 6 by the following procedure.

(Definition symbols) N: Length of symbol string for unknown speech data 01 First symbol of unknown speech data symbol string M: Phoneme HM to be matched
Number of states of M a (i, D: Transition probability of the arc connecting state i and state j in the phoneme HMM to be matched b (i, j, k): Phoneme HM to be matched
Probability that the arc connecting state ] and state j in M outputs the symbol k (initialization) P (0,0) = 1. 0 P (0,j)=1. Oe--(j=1--・M)P
(i, 0)=1. Oe-” (i=1・-N) (recurrence calculation (1=1...N, j=1...M))P (i,
j) = P (i-1, j) xa (j, j
)xb(j,j,Oi)+P(i-1,j-1)Xa
(j-1, j) xb (j-1, j, Oi) Q (
i)=P (i, M) (j=1...N) The results of phoneme matching are found in probability tables Q(1)...Q(N).

The HMM-LR method handles and analyzes the LR method and HMM method as described above in a unified manner. The HMM-LR method uses LR
The child table predicts phonemes in uttered audio data, and calculates the existence probability of the predicted phonemes by driving HMM phoneme matching for the predicted phonemes. This allows speech recognition and language processing to proceed simultaneously. Therefore, highly reliable and efficient processing can be performed without intervening intermediate data that bridges speech recognition and language processing. Hereinafter, the HMM-LR method will be simply referred to as a parser.

The parser grows several possible parse trees simultaneously. A parse tree represents a sentence as a one-dimensional word string, and represents the relationships between these words like a tree. A parse tree is assigned a probability value that the parse tree will be accepted, and if this probability value falls below a predetermined threshold, the parse tree is considered not worth growing and is rejected. . The parser has several locations to remember information about the parse tree it is currently growing. This location will be referred to as a cell below. One parse tree corresponds to one cell. A cell corresponding to a parse tree that has been accepted up to now is called an active cell. The information stored in the cell includes the following:

(1) LR parser state stack (2) Probability table Q calculated in the previous phoneme matching (
1) Value of Q (N) where N is the length of the symbol string for the input audio data.

There is only one cell C when parsing starts, and state 0 is bushed at the top of the state stack of the LR parser for that only cell C.

In addition, the following values are entered into the probability table Q of this cell C as initial values.

Q (0) = 1. 0 Q (i)=1. Oe--(i=1・N) Next, the parser selects one active cell, reads the state S at the top of the LR state stack of that cell, and examines the operation table corresponding to the LR child table state S. . The selected action is 5HI
If it is FT, the input symbol A to be 5HIFT is HM
M phonemes are matched, and the value of the probability table in the cell is updated as follows.

(Gradual calculation) P (0, D = 1. Oe-''' (j=1-M-
)P (i, 0)=Q (i) (i=1...
・N) P (i, D = P (i-1, j) xa
(j, j)Xb (j, j, Oj) +
P (j-1, j-1) xa (j 1.D
b (j-1, j, 01) (i=1...N, j=1・
・・M−)Q (i) =P (i, M−)
(j=1...N) However, M' is the number of states in the HMM of symbol A, and the probability table Q(1) updated by the above calculation.
0. .

Q (i) with the highest probability value among Q (N) is
If it is less than the threshold, this cell is discarded, and if it is less than the threshold, a new state is placed on the LR state stack.

On the other hand, if the selected action is REDUCE, the reduction action according to the grammar rules is executed. This is exactly the same operation as a normal LR parser. Also, the selected action is ACC
If it is EPT and all input audio data has been processed, the analysis ends.

Now, as a method to uniformly describe the phoneme model used in speech recognition using environmental information around the phoneme, phoneme environment clustering (phoneme environment clustering) is proposed.
Clustering (PEC). this is,
This is a method for extracting environment-dependent phoneme clusters by minimizing the total amount of distortion in the mapping between the phoneme pattern space and the phoneme environment space. Phonemic environmental factors include phonemic context,
Examples include pitch, power, speech rate, and language. Among these, phoneme context information is considered to carry particularly important information regarding the phoneme environment. In particular, H
When the phoneme context is known as in the MM LR method, P
Highly accurate recognition can be expected using the phoneme model with a high degree of phoneme separation obtained by EC.

[Problems to be Solved by the Invention] By the way, when phoneme context factors are taken up as the phoneme environment, the phoneme model determined by phoneme environment clustering becomes a model that depends on the phoneme context. In order to drive this phoneme model to perform continuous speech recognition, the parser must operate depending on the phoneme context. but,
The LR parser in the conventional HMM-LR method cannot operate according to the phoneme context, and cannot drive the above-mentioned phoneme context-dependent phoneme model.

Therefore, the main objective of the present invention is to provide a continuous speech recognizer for driving a phoneme context-dependent phoneme model.

[Means for Solving the Problems] The present invention is a continuous speech recognition device, which includes an HMM phoneme matching unit that calculates a probability for each voice of input speech;
The prediction LR parser section includes a predictive LR parser section that uses LR child table action specification items for phoneme prediction, and a phoneme context prediction section that predicts the phoneme context around the predicted phoneme using the LR child table action specification items. The phoneme context prediction unit predicts the phoneme context of the predicted phoneme, drives a phoneme context-dependent phoneme model that matches the environment of the phoneme context, and calculates the existence probability of the phoneme predicted by the phoneme context prediction unit. The information is determined by driving the HMM phoneme matching section.

[Operation] The continuous speech recognition device according to the present invention predicts a phoneme in input speech data using an LR child table, and predicts a phoneme context around the phoneme for the predicted phoneme using a phoneme context using an LR child table. By making these predictions in the prediction unit and verifying these predictions with the phoneme matching function of the HMM phoneme recognition unit, we can create a phoneme context-dependent HMM according to the LR parser phoneme context.
It is designed to drive a phoneme model.

[Embodiment of the Invention] FIG. 1 is a schematic block diagram showing the configuration of an embodiment of the invention. First, the configuration of an embodiment of the present invention will be described with reference to FIG. An audio signal is applied to an HMM phoneme matching section 101 via an input terminal 100 . The HMM phoneme matching unit 101 uses a phoneme context-dependent HMM phoneme model 10.
2 to match the phonemes. The phoneme-dependent LR parser section 109 includes a phoneme context prediction section 107 and a predictive LR parser section 1.
08. The predictive LR parser unit 108 predicts the next phoneme from the LR table 106, and the predicted phoneme is provided to the phoneme context prediction unit 107. The phoneme context prediction unit 107 sets the predicted phoneme as the central phoneme and calculates the LR
The phoneme context of the predicted phoneme is predicted by referring to the history information of the phoneme context written in the child table cell.

Furthermore, after determining the phoneme environment cluster that matches the predicted phoneme context described above, the control signal is
The HMM phoneme matching unit 101 is activated by the HMM phoneme matching unit 101 .

The HMM phoneme matching unit 101 drives a phoneme context-dependent HMM phoneme model corresponding to the phoneme environment cluster, and performs phoneme matching for the predicted phoneme.

The matching result 104 for the predicted phoneme by the HMM phoneme matching unit 101 is returned to the predictive LR parser unit 108. Prediction L
The R parser unit 108 repeats similar operations until it finds the ACCEPT operation in the LR child table 06. Then, a recognition result 109 is output from the predictive LR parser section 108.

FIG. 2 is a flowchart for explaining the specific operation of an embodiment of the present invention, and FIG. 3 is a diagram for explaining the operation of phoneme context prediction.

Next, with reference to FIGS. 2 and 3, the specific operation of one embodiment of the present invention will be described.

First, the information stored in the cell includes the following as shown in FIG.

(1)' LR parser state stack (2) Phoneme environment cluster stack (3) Probability table Q calculated in the previous phoneme matching (
1)...Value of Q (N). Here, N is the length of the symbol string for the input audio data.

As shown in Figure 2, the steps (
In the SPI (abbreviated as SP in the figure), there is only one cell C, and the state O is bushed at the top of the state stack of the LR parser of the only cell C. In addition, the following values are entered into the probability table Q of this cell C as initial values. Q (0) = 1. 0 Q (i)=1. Oe-” (i=1-N) In step SP2, the predictive LR parser 108 determines whether there is an active cell, and if not, ends the analysis, and if so, selects one active cell in step SP3. , the top state S of the LR state stack of that cell
, and check the action column corresponding to state S in LR child table 06. Then, the predictive LR parser unit 108 makes copies of the cell as many times as there are actions in the action column. The copy of the cell that is created is used to perform an operation, and the following operations are performed on this copied cell.

In step SP4, it is determined whether or not there is a cell created by copying. If not, the process returns to step SP2, and if so, the process proceeds to step SP5.

In step SP5, the operation corresponding to each cell is checked, and if the selected operation is 5HIFT, step S
Proceed to P6. Above steps SPI to step SP5
Up to this point, the processing is exactly the same as the normal HMM-LR method.

In step SP6, it is determined whether or not the input symbol A to be subjected to 5HIFT is allowed to be followed in terms of phoneme context in the referenced environmental cluster by referring to the top level of the environmental cluster stack in the cell. If a successor is not allowed, the cell is discarded in Ste-Tsubu SP7,
If the successor is permitted, the process advances to step SP8. In step SP8, a hypothesis of the predicted phoneme context is established as follows. Here, the predicted phoneme context will be explained as being usually made up of three factors: a preceding phoneme, a central phoneme, and a subsequent phoneme.

First, the next state S′ to be 5HIFT from the current state S
With reference to the operation in , the input symbol B of the item that is the 5HIFT operation is set as the predicted subsequent phoneme to the input symbol. 5HIFT in the next state S' to be SHIFT here
Skip if there is any other action. Next, the input symbol C at the top of the stack of input symbols in the cell is set as the preceding phoneme, and the preceding phoneme C and the predicted succeeding phoneme B described above are
A predicted phoneme context is generated using a phoneme triplet of the input symbol A and the above-mentioned input symbol A. A phoneme environment cluster is determined based on the phoneme context.

In step SP9, the input symbol to be subjected to 5HIFT is phoneme-matched by the HMM phoneme matching unit 101 using the phoneme context-dependent 8MM phoneme model corresponding to the determined phoneme environment cluster described above.

At this time, the update calculation of the value of the probability table in the cell is exactly the same as the update calculation of the normal HMM-LR method described above. Probability table Q(1) updated by this calculation...
In step 5 PIO, it is determined whether Q (i) having the highest probability value among Q (N) is smaller than a threshold value.

If Q(i) with the highest probability value is less than a threshold, this cell is discarded in step SPI 1 and becomes inactive. However, if it is smaller than the threshold, a new state is stacked on the LR state stack in step 5P12, and the determined phoneme environment cluster described above is stacked on the stack of environment clusters. In this case the cell remains active. Next, the process proceeds to step 5P13, and it is determined whether or not there is a SHI FT operation in the next state S- referred to in the LR child table 06. If it exists, the process returns to step SP6, and if it does not exist, the process returns to step SP2. Return to

On the other hand, in step SP5 described above, if the selected action is REDUCE, the process proceeds to step 5PI4, where a reduction action according to the grammar rules is executed.

This is exactly the same operation as a normal LR parser.

At this time, the cell remains active. Further, in step SP5, it is determined that the selected action is ACCEPT, and in step 5P15, it is determined whether all the input audio data has been processed,
If everything has been processed, the analysis is deemed successful and ends. Otherwise, this cell is discarded in step 5P16 and the process returns to step SP2.

[Effects of the Invention] As described above, according to the present invention, it is possible to realize an LR parser that operates according to phoneme context in continuous speech recognition using the HMM-LR method, and to use the phoneme context-dependent LR parser. It is possible to drive phoneme context-dependent phoneme models.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram of an embodiment of the present invention. FIG. 2 is a flow diagram for explaining the operation of one embodiment of the present invention. FIG. 3 is a diagram for explaining the operation of predicting phoneme context. FIG. 4 is a diagram showing an example of grammar rules. FIG. 5 is a diagram showing an example of converting a grammar rule into an LR child table. FIG. 6 is a diagram showing an example of the HMM. In the figure, 100 is an input terminal, 101 is f (MM phoneme matching unit, 102 is phoneme context dependent type) (MM phoneme model, 1
06 is the LR child table 107 is the phoneme context prediction unit, 108
109 indicates a predictive LR parser section, and 109 indicates a phoneme context-dependent LR parser section. Patent Applicant A.T.R. Automatic Translation Telephone Laboratory Co., Ltd. Figure 3 LR Table Operation Table Figure 4 1
-(1) 5TART -NP VP(2)
5TART -VP (3) NP -N (4) NP -NP (5) VP→okur e(6)
VP-ku r e(7)N -4k
ane (8) P -0 111-苧--------Ita---------
--Figure 6 bulk b12k b23 Figure 5 ACT I ON table status *itrgoenuS Os2 ss l r3 s72
slOs9 3 "2 4 sl I s
56
ac
c7 "8 8 r4 10
sl 5II
5912 rl +3
51614
sl7Is
s18+6
51917 r6 18 r7 20 rs GOTO table

Claims (1)

[Claims] HMM that calculates the probability for each phoneme of input speech
(HiddenMarkovModel) A phoneme matching unit, a predictive LR parser unit that uses action specification items in the LR (Left to Right) table for phoneme prediction, and a phoneme context around the phoneme predicted by the predictive LR parser unit to perform actions in the LR (Left to Right) table. a phoneme context prediction unit that makes predictions using specified items; and driving an HMM phoneme matching unit that matches phonemes using a phoneme context dependent HMM phoneme model that matches the phoneme context predicted by the phoneme context prediction unit. A continuous speech recognition apparatus, characterized in that the existence probability of the phoneme predicted by the predictive LR parser section is determined by: