“Computation in Linguistics: A Case Book” | Open Indiana

Bernard Spolsky

1.0 BACKGROUND

While the efficacy of programmed instruction as a teaching tool has already been amply demonstrated, there remain many questions as to the most effective form of presentation. Programs may be organized in many different ways (the technical term used for such organization is a paradigm), and may call for various types of responses from the student. This paper presents a programming strategy that will allow students to construct their responses and will provide different branches for different responses.

1.1 Paradigms. The two main classes of paradigms are linear and branching, with the possibility of various degrees of combination of the two extremes. A purely linear program is one in which the frames or items follow in an inflexible order; the student works through them one after the other irrespective of his ability, the accuracy of his responses, or any other individual differences. In a pure branching program, the order in which the frames are presented depends entirely on his responses; a mechanism of some kind decides, on the basis of each response (or of the cumulative history of a number of responses), which frame he should do next.

There are strong arguments in favor of branching. Even if one does not accept the Crowderian model,¹ it is often considered valuable to allow for some degree of branching within a basically linear program (e. g. on the basis of criterial test items, to shunt the student onto a faster or slower program track, or to present more practice items in a matter that has not been mastered). A branching program allows for differences between students; a linear program forces all students to follow the same track. If a linear program is aimed at the slower student, and is successful in ensuring that the student is correct in at least 90 per cent of his responses,² the brighter student is almost certain to be bored and to find the program insultingly simple-minded; if it is aimed at the brighter student, it is probable that the slower student will find many of the steps too large and will consequently make so many erroneous responses that the effective working of the principle of reinforcement of desired behavior will be destroyed. There can be no happy medium, for there will always be students who are brighter or slower than those for whom the program was constructed.³ The only solution to this dilemma is the provision of the maximum possibility of program branching on the basis of student responses.

1.2 Response modes. The three main types of student response are covert, overt constructed, and overt multiple-choice. In the constructed response mode, which is the form favored by those who follow Skinner most closely, the student makes up his own answer, writing or speaking it as the program directs.

In the covert mode, the student simply reads the text; he need make no overt response, for there are no blanks. The program is however essentially the same as a constructed response one with the blanks already filled in. For example, a constructed response frame might appear like this:

The underlined word in ‘He ate an apple’ is a ___. verb In a reading program, this would appear as:

The underlined word in ‘He ate an apple’ is a verb.

In the multiple-choice mode, the student selects one of a number of possible responses offered to him. A frame in this form would appear like this:

What part of speech is the underlined word in the sentence

‘He ate an apple. ‘?

a. verb

b. noun

c. adverb

d. pronoun

While there is no conclusive evidence as to which type leads to the most efficient learning, it is a reasonable assumption that there are areas in which constructed responses are essential. Foreign language teaching is such an area, for there the aim is often to teach the student to recall as well as to recognize correct responses. Even if we accept the hypothesis that training the student to discriminate correct and incorrect forms will transfer to his being able to produce correct forms, there is a stage at which the student must be able to make up his own responses, and it is these constructed responses which are important in deciding how well he has mastered the material that has been presented so far.

The present state of techniques in programmed instruction permits the use of any response mode with linear programs, but only of multiple-choice responses with branching programs. This paper shows how it is possible to use a computer in order to build a branching program that uses the constructed response mode.

2.0 THE PROBLEM

The crux of the problem that has been described is how to build into a program a device that will analyze the response that the student has made up, and, on the basis of this analysis, decide which branch, or which frame, is to be presented to the student next. The programming logic implied is shown in Figure 12.1. The key problem is in Box 3, where the response is analyzed, and Box 4 where a decision must be made as to which frame should be next presented.

Let us consider some possible solutions. In a program that uses multiple-choice responses, the solution is relatively straight-forward. Consider Example 1.

Example 1. A branching multiple-choice frame

Make this sentence negative: We went.

a. We went not	(Go to frame 24)
b. We did not go.	(Go to frame 35)
c. We did not gone.	(Go to frame 15)
d. We go not.	(Go to frame 45)

In this example, it will be seen that the solution has been to provide each of the choices offered with the address of the frame that is its appropriate successor; the correct answer will of course carry on with the main line of the program, while the distractors will lead to suitable remedial branches. This system is easily automated,⁴ but it is to be noted that it involves two serious limitations:

1)It calls for the student to recognize the correct answer rather than to produce it.

2)It identifies only those errors whose possible occurrence has been predicted by the programmer when he made up the distractors. In Example 1, for instance, there is no allowance for the student’s tendencies to make up such responses as We not went or We did not went.

Another solution is to have frames in which branching is decided by treating a constructed response as being either right or wrong. Such a frame is illustrated in Example 2.

Example 2. A branching either-or constructed response frame

Make this sentence negative: We went.

Correct response: We did not go.

If correct, go to frame 30.

If wrong, go to frame 15.

In this example, the student makes up his response which is then compared with the correct response. If he is right (and some degree of tolerance of, for example, spelling may be built in), he continues with the main track; if wrong, he may be shunted onto a remedial loop or returned to go through the previous section again. Such a frame is used in many linear programs. It is also the type that has been used in reported computer-based programs because it lends itself to relatively straight-forward programming. In the IBM project, for instance,⁵ the author of a program specifies not only the answer or answers which he will accept as correct, but also any student errors that he anticipates. In this way, provision is made for dealing with predictable wrong answers. But it is obvious that this solution too remains severely limited in its ability to deal with the great variety of erroneous responses that the student may make.

The solution offered in this paper is a program that will carry out the following steps:

1)Analyze any response that the student may construct;

2)identify as precisely as possible all errors there may be in it;

3)inform the student of the nature of his error; and

4)select and present appropriate remedial work.

Such an operation approximates most closely to the behavior of a good tutor; it can be carried out with the aid of a computer, and its feasibility is primarily a problem of computational linguistics.

There are a large number of other advantages which accrue when a computer is used to control programmed instruction; in this paper, attention will be paid to the possibility of the automatic generation of frames, and mention will be made of the value of the computer for gathering statistics and for modifying the operation of a program in the light of the cumulative history of the experiences of the student working on it. However, it is clear that one of the main obstacles to developing a program that will give the maximum attention to individual differences has been the lack of a method of dealing satisfactorily with unpredicted erroneous responses. In this paper, it is shown that this obstacle can be overcome by using a suitable method of analyzing the material to be taught.

3.1 Scope of the program. The preparation of material to be presented through the medium of programmed instruction calls first of all for the careful definition, in explicit and operational terms, of the terminal behavior that the program is designed to shape. When it is desired to make use of a computer to control the presentation, this definition must be even more precise than usual, for it will have to be programmed for computer recognition. In this paper, therefore, an area to be taught has been selected that can be more or less exhaustively described.

The program outlined in this paper is one that will form part of a course in English as a foreign or second language; it tests and teaches the formation of English verb phrases. For the purpose of the program, the verb phrase is defined as consisting of a subject pronoun (which is optional), one or more (optional) auxiliary verbs, and a verb base. It thus covers phrases ranging in complexity from Eat! to Might he not have been being eaten? .

This program is intended to be preparation for one in which the student will be taught the appropriate uses of the tenses that he has here learned to form. In other words, at the present stage he will form the phrase in response to a specific instruction such as ‘Make this sentence past’; in the projected continuation, the stimulus will be something like ‘Add the word yesterday to this sentence.’

The terminal behavior that the program aims to shape may be specified as follows:

When the student is presented with an instruction and an English verb phrase, he will respond with an appropriate adaptation of the verb phrase in accordance with the instruction he has been given.

For example, when he is presented with the instruction ‘Make this sentence negative: We went.’, he will respond We did not go. The full repertory of instructions is detailed in Table 12.2.

The analysis of the English verb phrase on which the program is based is set out in the state diagram shown in Figure 12.2.⁶ Following this, it is possible to consider any verb phrase as consisting of the following elements:

1. Subject pronoun

2. Concord marking

3. Modal auxiliary

4. Perfect auxiliary

5. Continuous (progressive) auxiliary

6. Passive auxiliary

7. Preterit (past) marking

8. Negative marking

9. Question marking

10. Dictionary item (base form).

3.2 Conditions. It has been assumed that the student will type his responses into a console connected with the computer. Frames will be either typed out on the same console or displayed by some other device. The student’s ability to use a keyboard is therefore assumed.

4.0 PEDAGOGICAL STRATEGY

One of the greatest values of programmed instruction is that it can be used as an instrument of research into the value of different teaching methods, to the extent that it makes it possible to control many variables in the learning situation and provides a clear and precise record of how certain material has been presented to the learner. This program has been planned to take full advantage of this point; the various strategies that have been built into it may easily be modified if it is desired to test their relative effectiveness.

The decision to build a remedial program rather than one which presents totally new material has been arrived at for two reasons:

1)Most programmed work in foreign languages has been concerned with the beginning stage where it is somewhat easier to define the starting point and the order of steps that are to be followed. This program can give some idea of how more advanced work may usefully be supplemented and accompanied by a series of small selfcontained programmed units.

2)The remedial framework is the one that calls for the maximum use of branching based on the analysis of the responses that the student has made. It would be relatively simple to tie the analysis subprogram into an initial teaching program, adding limitations as to the remedial subroutines to be used at various stages.

The pedagogical strategy of the program may be considered as consisting of four stages, the decision logic being shown in the flowchart in Figure 12.3.

STAGE I. Diagnostic frames (Box 1) The main track of the program consists of a number of diagnostic frames which test the student’s ability to carry out all the operations in forming English verb phrases that the program deals with. At this stage, the student is presented with a frame of the type shown in Example 3.

Example 3. Sample diagnostic frame

FRAME NUMBER 1

MAKE THIS SENTENCE NEGATIVE: WE WENT.

The student types his response; if it is correct, the next diagnostic frame is presented. In the present state of the program, a set of diagnostic frames has been prepared manually, but this manual preparation will be replaced by the automatic frame generation procedure described below (section 5.32).

STAGE II. Try again (Box 3) If the analysis routine discovers that there is an error in the student’s response, he is asked to try again. This stage is included in order to allow the student to correct any errors he may have made in typing; it will also deal with spelling errors. If the second try is correct, the next diagnostic frame is presented. A statistics program, not considered here in any detail, would be included to keep track of the number of second tries that a student has made.

STAGE III. Cued response (Box 5) If, when it is analyzed, the second try is found to still contain an error, the student is informed of the type of mistake that he has made. The program will print out a clear cue such as:

YOU HAVE MADE AN ERROR. YOU HAVE NOT MADE THE SUBJECT AND THE VERB AGREE. TRY AGAIN.

The statistics program would keep a record of such cues, and would give extra remedial work when repeated cues have been needed for the same error.

STAGE IV; Remedial routine (Box 7) If the student still makes an error after a cue has been presented, or if it has been necessary to give him repeated cues for the same error, the program will present a remedial teaching routine which will show him how to form the correct response and which will give him practice in the appropriate form. An example of a remedial routine is given in detail later. The last step of this stage will be to present again the diagnostic frame in which the error was first made.

5.0 DESCRIPTION OF PROGRAM OPERATION

The program as a whole consists of the following sections:

1. Main routine

2. Analysis of student response

3. Preparation of frames

4. Remedial routines

5. Remedial frame generation

6. Statistics

5.1 Main routine. The decision logic for the main routine is shown in the flowchart in Figure 12.1.

5.20 Analysis of student response. The central part of the program is that part which analyzes the response that the student has typed, notes any errors that have been made in the formation of the verb phrase, and then compares the student’s response with the correct response for the frame. This process involves coding the student’s response into the format referred to in section 3.1 above. Example 4 shows a coded verb phrase.

Example 4. A coded verb phrase

We have not been asked

1. Subject pronoun: Plural

2. Concord marking: Plural or 1st person singular

3. Modal auxiliary: 0

4. Perfect auxiliary: 1

5. Continuous auxiliary: 0

6. Passive auxiliary: 1

7. Preterit marking: 0

8. Negative marking: 1

9. Question marking: 1

10. Dictionary item: Past participle of ASK

The analysis falls into a number of subroutines described in the following sections.

5.21 Abbreviations. (For flowchart, see Figure 12.4) The student’s response is scanned for apostrophes; if they are found, abbreviations are resolved using a dictionary that lists acceptable forms and common errors.

5.22 Tagging pronouns. (For flowchart, see Figure 12.4) The sentence is now scanned for a word on the pronoun list (HE, SHE, IT, WE, THEY, YOU, and I); any pronoun found is tagged with the appropriate concord marker (third person singular, plural, or first person singular). Absence of a pronoun is interpreted as a mark of the Imperative.

5.23 Negatives and questions. (For flowchart, see Figure 12.4) The student’s response is next scanned for the occurrence of the word NOT; if it is found, the response is coded as ‘Negative marking: 1.’ The position of the tagged subject pronoun is now checked; if it is found to be the second word in the sentence, the response is coded as ‘Question marking: 1.’ If the pronoun is found to be in neither initial nor second position, a notice of student error (Student error - Ell, misplaced subject) is recorded.

5.24 Main look-up. Not and the subject pronoun having been tagged as no longer needed, each of the remaining words in the student’s response is looked up in the verb-form dictionary. This dictionary contains all the forms of the auxiliary verbs that are dealt with by the program, each marked with an appropriate grammar code (see Table 12.1). The grammar code marks the form as being an auxiliary; identifies it as a part of one of the verbs BE, HAVE, DO, CAN, MAY, MUST, SHALL, or WILL; notes restrictions on its use in this form (e. g. AM is only used with the first person singular; HAD is to be treated as a preterit of a past participle); and gives restrictions on what suffixes may legitimately be attached to it (e. g. AM may not be found with ing, s, or ed; BE may not be found with ed or s). Also in the verb-form dictionary will be found any forms of the verb or verbs that have been fed in with the frame (see section 5.3 below on the preparation of frames). These forms are similarly marked with a grammar code which here concerns only restrictions on use and suffixes.

The main look-up routine is shown in the flowchart in Figure 12.4. As a word is looked up, its grammar code is entered in a numbered coded word location, the first word going in Coded Word Location I, the second in II, etc. If a word is not found in the verb-form dictionary, it is checked for possible suffixes.

First, an S-stripping subroutine (shown in the flowchart in Figure 12.4) recognizes verbs that are in the third person singular form. This subroutine allows for various spellings of the third singular present marker; it includes the possibilities of recognizing two misspellings, the use of an S-form in the wrong position (Student error-E21), and the use of an S-form in a question or a negative sentence (E42). If the marker is found, the code ‘Third person singular’ is entered under verb concord of the Coded Word Location.

If the word that has not been found in the verb-form dictionary-does not end with S but with ED, an ED-stripping subroutine (in Figure 12.5, distinguishes between preterits and past participles,⁷ recording notices of any errors that the student may have made in spelling or in formation, and deciding the appropriate information to be entered in the Coded Word Location.

Finally, an ING-stripping subroutine deals with any word not so far identified; giving notice of wrongly formed present participles, and of identifiable and unidentifiable spelling errors.

As a word is identified, its grammar code, sometimes as modified by the stripping subroutine, is entered in the next empty numbered coded word location.

5.25 Subject-verb concord. In the next stage (shown in the flowchart in Figure 12.5), the information in the first Coded Word Location relevant to concord is entered in the coding of the response; the concord marking of the pronoun and the concord marking of the verb are now compared, any discrepancy being noted as an error. All Coded Word Locations are now checked to see that there is no word in an auxiliary verb position that is not marked as an auxiliary verb, appropriate error notices being recorded if necessary.

5.26 Auxiliary verbs. Each of the Coded Word Locations that have been used is now checked in numerical order, and appropriate information is recorded in the coded response location and in the notice of student error location. The coded response location has the form shown in Example 4 above. The recognition of a modal verb leads to its presence being noted in the Modal auxiliary column; a check is then made to be sure that the word in the following Coded Word Location is a base form. Similarly, subroutines deal with the Perfect, the Continuous, and the Passive auxiliaries. Finally, a subroutine deals with the occurrence of the auxiliary do. Notices of student errors detail incorrectness of position or use of incorrect forms after an auxiliary verb.

5.27 Comparison with correct response. The student’s response has now been completely coded into the format shown in Example 4, the information having been entered in the coded response location. The contents of the coded response location may now therefore be compared with the correct response code, any discrepancies leading to the appropriate error notices.

5.30 The preparation of frames. The diagnostic frames referred to above (section 4, State I) may be prepared manually. The format of a frame is shown in Example 5.

Example 5. Format of a typical diagnostic frame

Dictionary item with its grammar code: ASK base form

Remarks: Go to next diagnostic frame.

Manually prepared frames such as that in Example 5 could be checked, by item analysis, for their usefulness, and a set of them could form a diagnostic program.

5.31 Automatic generation of frames. While the program will be able to function satisfactorily with a set of manually prepared frames, its value can be greatly increased by making it generate its own diagnostic frames. This will mean, firstly, that the student will not be limited to using a finite set of prepared frames, but can, if necessary, come back and use the program again and again; and secondly, as will be shown below in section 5.50, the generation strategy proves to be an effective way of dealing with the problems of the remedial subroutines.

The basic concept on which this automatic generation is based is the use of a randomly-generated code number, the interpretation of which is restricted in certain ways.

A frame generation program consists of three main parts:

I.The first part of a frame generating program is a store of randomly-generated numbers, each consisting of 36 binary digits. This number of digits suffices for the purposes of the present program, but more would be needed for a more sophisticated program. The interpretation of these digits is shown in Figure 12.6.

II.The second part will be a subroutine (shown in the flowchart in Figure 12.7) which takes the first six digits, looks them up in the instruction code (see Table 12.2), and sets certain of the other digits as determined by the code list. For example, if the first six digits are 000000, the instruction code interprets this as ‘Make this sentence negative:’; the digit in the random number that will be interpreted as ‘Negative’ for the correct response code is therefore set at ‘1’, while the digit that will be interpreted as ‘Negative’ for the content generating code is set at ‘0’. No other digits will be changed unless some other restriction has been imposed for this stage of the student’s instruction.⁸ Table 12.2 gives full details of the alterations that will be made to the randomly-generated numbers.

III. The third part of the frame generation program generates the frame as follows:

1.It allocates a number to the frame and prints it out.

2.It prints out the instruction that was looked up in the previous part.

3.It generates and prints out the content.⁹

4.It takes the appropriate digits in the randomly-generated number (as modified by the previous part), and prepares the correct response code.

5.It adds the dictionary item that has been used for the frame, together with its grammar code, to the verb dictionary.

6.It adds remarks to the frame giving the address of the next frame to be presented after this one.

5.40 Remedial routines. When any notices of student errors have been recorded, they are first ordered and then treated in accordance with the strategy explained in section 4 above; the student is first asked to try again, then given a precise cue, and finally a remedial teaching program is presented. A remedial teaching program consists of a number of manually prepared frames (see example 7) and of instructions to generate other frames (see next section).

5.50 Remedial routine generation. As has been seen, the basic frame generation program described above introduced the concept of combining the use of a randomly-generated number with certain restrictions on its interpretation. This concept is carried on in the remedial routine generation program, where the same randomly-generated numbers are used, but added restrictions are placed on their use. Thus, the remedial frames include not only these restrictions, but also preset remarks which control the path to be followed before returning to the diagnostic frame or dealing with other errors. The working of this principle is made clearer if one considers the specifications for one such remedial teaching routine. Each of the following sections details a part of this routine; it will be seen that not all routines will include all the parts.

5.51 Precise identification of error. In certain cases, it is possible to identify the error with more precision than has been done by the main analysis routine. The main program, for instance, identifies a student error in concord (Student Error-E32, subject and verb do not agree in person and number). A short subroutine (shown in the flowchart in Figure 12.8) decides which of six possible incorrect combinations has been used, recognizing for instance that a third person singular verb has been used with a first person singular subject pronoun, and gives the address of the appropriate remedial teaching routine.

5.52 Further testing. It is next advisable to check that the identification has been right; that the student’s error has arisen from the precise problem that has been identified rather than from a combination of elements. It is also desirable to know how generalized is the error. One or more frames are therefore presented which will call on the student to carry out the instruction in isolation. The specifications for this part of the routine are given in Example 6; it will be seen that this sample routine consists in effect of instructions to generate frames from randomly-generated numbers that have to be modified to suit the special purposes of the frames.

Example 6. Three sample frames for testing of error

Address of this routine: E3222 (subject is plural, verb is third person singular)

Generate remedial frames, from randomly-generated numbers, subject to the restrictions detailed:

Frame number: E3222 1

Instruction set as: 10100 (Interpretation: Replace the subject of this sentence by the word they)

Content: Set all modals and auxiliaries at 0. Preterit 0.

Correct response: Modals, auxiliaries, Preterit all 0.

Remarks: If right, print ‘RIGHT. AND THIS.’ and then go to E3222 2.

If wrong, print ‘NO. LOOK AT THIS.’ and then go to E3222 4.

Frame number: E3222 2

Instruction set as: 10011 (Interpretation: Replace the subject of this sentence by the word ‘we. ‘)

Content and Correct Response: Modals, auxiliaries, preterit, all 0.

Remarks: If right, print ‘RIGHT. AND THIS.’ and then go to E3222 3.

If wrong, print ‘NO. LOOK AT THIS.’ and then go to E3222 5.

Frame number: E3222 3

Instruction set as: 10101 (Interpretation: Replace the subject of this sentence by the word you.)

Content and Correct Response: Modals, auxiliaries, preterit, all 0.

Remarks: If right, print ‘GOOD. THE LETTER “S” IS NOT ADDED TO FORMS OF THE VERB AFTER THEY, WE, YOU. NOW TRY THIS AGAIN.’ and present the original diagnostic frame again. If wrong, print ‘NO, LOOK AT THIS’ and go to E3222 6.

It will be seen that the first of the frames in Example 6 calls on the student to replace the word he by the word they in a sentence that contains a simple tense form. If he does this correctly, the next frame will call for the use of we and the third for you. The third frame, if answered correctly, will include a restatement of the rule that has been broken in the diagnostic item. Each of these identification frames has, as its address in the case an error is made, a remedial instruction frame.

5.53 Remedial teaching. We have seen in the previous two sections that the error noted in the main analysis routine has been identified as precisely as possible, and that the program has decided which rule the student failed to apply. The remedial teaching frames are now used to present this rule and to drill the student in its application. The specifications for this part of the program are illustrated in Example 7.

Example 7. Three sample teaching frames

Frame number: E3222 4

Print out: HE EATS.

THEY EAT

WHAT LETTER DOES THE VERB AFTER HE END WITH? (Allow space for student to respond)

IT ENDS WITH THE LETTER S. DOES THE VERB AFTER THEY END WITH THE LETTER S? NO. THE LETTER S IS NOT ADDED TO FORMS OF THE VERB AFTER THE WORD THEY. Go to E3222 1.

Frame Number: E3222 5

Print out the same material as in E3222 4, substituting WE for THEY.

Go to E3222 2.

Frame Number: E3222 6

Print out the same material as in E3222 4, substituting YOU for THEY.

Go to E3222 3.

These remedial teaching frames may take the form, as was shown in Example 7, of simple linear constructed-response items, or they could be built as multiple-choice items with branching for incorrect replies. However, to have remedial teaching frames that call for constructed responses that will need to be analyzed by the main analysis routine would call for more complication than is justified in the early stages of the development of the program.

5.54 Practice. The final section of a remedial routine is a number of practice frames. In the example given, these are the same in form as those that were used to identify the error, but the random generation will ensure some variation in content. It will be noted that the final frame of a remedial routine states the rule that is in question and then calls for the diagnostic item in which the original error was made to be presented again.

5.60 Statistics. Reference has been made earlier to the possibility of building in a statistics program. At a more sophisticated stage, this could be used to control various parts of the main program, so creating a greater degree of program flexibility and allowing for individual student differences. In an early stage, however, it would be valuable to set up a program to collect the following information:

a. Information about the student (name, native language, standard).

b. A list of frames that have been presented to each student.

c. A list of errors that have been recognized by the program and that the student has corrected

1. on the second try;

2. when a cue has been given; and

3. after a remedial instruction routine.

d. A list of errors that have been recognized by the program but that the student has still not corrected after a remedial routine.

e. Details of remedial teaching routines that have been used.

f. A list of misspellings or of words that have been so labeled. This list, correlated with the information about the student’s native language and standard, would serve later as the basis for planning a way to build into the program the ability to offer remedial spelling teaching.

6.0 CONCLUSION

The aim of this paper has been to show that it is feasible to set up an automated teaching program that will permit branching on the basis of responses that the student constructs himself. The crux of the matter has been shown to be in the analysis program, and the problem has been shown to be solvable through a linguistic analysis of the material being dealt with. That is to say, given a workable method of analyzing input texts and coding their elements so that they may be compared with desired inputs, it is possible to use this process of analysis and comparison as a method of identifying errors that the student has made in his responses to diagnostic questions. It has also been shown that this analysis program lends itself to the requirements of a frame generation procedure that will form part of a program to prepare material that can be used to teach the desired behavior.

It will still be necessary to show that such a program as has been proposed here will be effective in use; such proof must wait until the program is tried out in practice.

It would also be most interesting to consider the theoretical implications of the program. There has been much debate on the criteria for a good grammar: frequently the term ‘pedagogical grammar’ has been loosely used to justify a grammar that cannot be rigorously defended. A sounder definition is that a pedagogical grammar is one that is to be used in teaching a language to a native or nonnative speaker. A pedagogical grammar would then be considered to consist of a number of linguistic statements,¹⁰ and its validity might be established by its ability to fulfill the requirements of a given language teaching program. This paper makes it clear that such a statement must make it possible to code any response that the student may construct to a given stimulus in such a way that it may be compared with the code of the correct response which is required and that specific errors in the student’s response may be unambiguously recognized, and it has shown that a linguistic statement of the form of the English verb phrase as set out in the paper does lend itself to such a purpose.

NOTES

1. The intrinsic program that Norman Crowder has devised is not in fact a pure branching paradigm, but rather is based on the idea of letting the student’s choice of answers to questions that are included in the text direct him to remedial material when necessary. See, for example, Norman Crowder, ‘On the difference between linear and intrinsic programming’, Phi Delta Kappan (March, 1963).

2. See James G. Holland and Douglas Porter, ‘The Influence of Repetition of Incorrectly Answered Items in a Teaching-Machine Program’, Journal of the Experimental Analysis of Behavior 4. 30 5307 (1961).

3. See John B. Carroll, ‘Programed Instruction and Student Ability’, The Journal of Programed Instruction, 2:4. 7-12 (1963).

4. As is done in Crowder’s AutoTutor. See Norman A. Crowder, ‘Automatic Tutoring by Instrinsic Programming’, in A. A. Lumsdaine and R. Glaser (eds.), Teaching Machines and Programmed Learning, Department of Audio-Visual Instruction, NEA, (1960).

5. See A. Maher, ‘Computer-Based Instruction (CBI): Introduction to the IBM Research Project’, IBM Research Report RC-1114 (March 6, 1964); also John E. Coulson, Programmed Learning and Computer-Based Instruction (New York, 1962).

6. This state diagram is of the type that has been developed by Jean-Paul Vinay. See Canadian Journal of Linguistics, 9: 1. 57 (1963).

7. This distinction cannot always be made for there is sometimes ambiguity. Ambiguities may be treated in two ways: each of the possible solutions may be accepted in turn and carried through until an error is found, in which case the other solution is tried, or a notice of ambiguity may be given, asking the student to say which of the possible interpretations he intended.

8. In the early stages of teaching, for instance, a restriction might be set so that the student would not have to deal with more than one or two auxiliary verbs in a sentence.

9. This section of the program was developed for a seminar on computational linguistics conducted at the Université de Montréal by Professor J. -P. Vinay. It was coded for computer operation, as part of the Language Data Processing Seminar, by James Stone.

10. See Albert Valdman, ‘Linguistic Statement and Language Teaching’, Proceedings of the Ninth International Congress of Linguists, Cambridge, Mass. 1962 (The Hague, 1964).

Table 12.1 Verb-form Dictionary