2. PROCEDURES FOR ADMINISTERING THE AKT
The AKT instruments are "standardized" tests. In order that the tests perform the various functions intended, it is imperative that they be administered under specified conditions, in appropriate surroundings and in accord with certain directions. The norms for the AKT series assume that these conditions of administration have been met. Given this, the norms yield dependable information concerning an examinee. Failure to follow the directions as described below may invalidate the use of the norms.
To ensure that the results yielded by the AKT instruments are accurate and meaningful, this part of the manual explains the directions for administering the instruments in detail.
The Institution Examiner and Proctor
The AKT instruments and their associated materials are distributed only to recognized anesthesia resident training programs, and only to a person within each such institution who has been designated as that institution's "Examiner." Because the AKT instruments are administered widely and on a continuing basis, maintenance of their security and the appropriateness of their administration in each setting are essential. Both instrument security and appropriateness of administration are the responsibility of the Institution Examiner.
The Examiner need not be the individual who actually administers the AKT in any given testing session. Optionally, the Examiner may appoint someone to act as Proctor. However, the Examiner should be familiar with the contents of this manual, and in particular the section dealing with the role of Proctor, before delegating the Proctor's role.
When a testing packet arrives at a participating institution, the Examiner should check the packet to be certain that the needed materials have been shipped. These materials should include:
1. An Administrator's Manual, if ordered;
2. The number of test booklets ordered;
3. A specimen answer sheet;
4. The number of answer sheets that were ordered (same as the number of test booklets);
5. A sufficient quantity of No. 2 soft pencils;
6. An itemized invoice; and,
7. A self-addressed return envelope.
Should any needed materials be missing, the Examiner should call Metrics immediately at (978)-256-9344.
To maintain the security of the AKT, lock up all tests in a safe place until they are to be administered.
Role of the Examiner/Proctor
Below is a set of guidelines for the "Institution Examiner" and "Proctor" (if applicable) concerning the administration of any of the AKT instruments. In this section, the person responsible for administering the test is referred to as the "Proctor."
1. The Proctor should become thoroughly familiar with the testing materials and the specified procedures before attempting to administer either examination. A sample completed answer sheet has been included in the shipped materials to show how the information grids on each answer sheet should be completed.
2. Examinees should be given ample advance warning that an examination will be administered. Under no circumstances should an examinee be required to respond to an AKT examination without having been properly informed concerning its administration, or without the requisite time in which to consider and respond to the test questions in an atmosphere relatively free from anxiety.
3. The AKT-1 is available in two forms (A and B) so that it may be given at the beginning and at the end of the first month of residence. Form A and Form B may be regarded as equivalent in terms of what they measure. One of the two forms has been designated as the pre-test instrument for your institution and the other, the post-test instrument. This has been done to ensure balance across the national sample. Test booklets are appropriately stamped on their front covers to indicate their proper use. Make certain that examinees who are to be pre-tested or post-tested use test booklets stamped for that purpose, and that they so indicate on their answer sheet as described in the section "Directions for Administration."
4. If the AKT-1 is administered either as a pre-test or as a post-test in association with a resident program in anesthesia, an examinee should sit for the examination within a few days of either commencing or completing his or her first month of residency training.
5. If the AKT-1 is administered as a pretest, examinees should be informed that its purpose is to determine their entry level knowledge, not their mastery of anesthesiology. Examinees should also be told that a post test will be administered at a later time.
6. If an examinee is to be given the AKT-6, make certain that the day set for giving the examination is as close as possible to the six-month point in the examinee's residence training.
7. The AKT-24 should be administered to CA-II residents toward the end of the 24th month of their residence.
Concerning Optically Scanned Answer Sheets
Examinees will record their responses on answer sheets that will be processed by optical scanning equipment prior to computer analysis. Optical scanning is accurate and fast, provided that examinees have properly filled out the answer sheet at the time the examination is given.
It is important that the instructions given are followed and that No. 2 soft lead pencils are used in completing the grids on the response sheet and in responding to the test items. Otherwise, an examinee's responses could be incorrectly entered into the computer file or an individual's identity could become obscure.
Optical scanning provides quick and accurate transfer of examinee recorded information to data useful for analysis by computer if certain guidelines have been followed carefully. The rules are as follow:
MARKING. The optical scanning equipment will only recognize marks that have been made using a soft lead pencil (No. 2 pencil). Any lighter pencil or any ball point pen marks entered will not be recognized dependably by the scanner. Thus, use of a proper pencil in completing the response sheet cannot be overemphasized. Also, the small circles used to record each response must be filled, not simply "Xed out" or checked. For proper instructions as to how to mark the response sheet, see its reverse side.
ERASING ENTRIES. Responses may be changed at the examinee's will. However, the prior response must be completely erased in order for the response change to be complete.
CONDITION OF TESTING MATERIALS. Examinees may make any marks they wish on the examination form. It may also be folded as examinees desire. The response sheet, however, must not be folded, stapled or ripped as doing so will hamper its passage through the scanning equipment.
IDENTIFYING INFORMATION. Special attention should be given to the response blocks that serve to uniquely identify each examinee and the examinee's institution, time of testing and form used. Examinees should be given ample instructions in filling out these blocks and sufficient time to do so accurately. A detailed guideline for this is provided later in this section of the manual under, "When Administering the Test."
Scheduling the Examination
The general rule for conducting an appropriate examination session is that the examinees be made comfortable, and have all necessary materials at their disposal.
Beyond these requirements, some additional considerations are important and should be listed.
1. Arrange for a suitable site at which the AKT may be administered. The site chosen should be free from distractions, provide ample room for each examinee to comfortably complete the examination, be well lighted and properly ventilated.
2. Arrange for the examination area to be free for two and one half hours (three hours if the AKT-24 is to be administered). Though the AKT -1 and 6 require two hours of administration time (two and one half for the AKT-24), an additional 15-20 minutes prior to administration will be needed in order that the answer sheets may be properly completed by each examinee and that unforeseen problems may be handled.
3. An examination should not be scheduled when insufficient time is available for completion.
4. Time limits given for the AKT should be strictly followed. Giving of more or less than the prescribed two hours for the examination will probably invalidate the norms for an examinee, as well as the evaluation information provided for him/her and the institution's program.
5. Examinees may leave the room when they have completed the examination. If all examinees who sit for an examination session have completed the AKT before the two hour time limit has expired, the session may be ended.
In preparing to administer the AKT, the Proctor for the examination session should check to be certain that the following materials are available:
1. A copy of the AKT examination book for each examinee (appropriately stamped for the time the examination is to be given i.e., either pre or post if administering the AKT-1);
2. At least one No. 2 soft lead pencil for each examinee (Note: colored pencils, hard lead pencils and pens of any color will not be recorded by the optical scanner);
3. These "Instructions for the Examiner/Proctor";
4. A watch or clock to be used in timing the examination session; and,
5. A sign which reads, "Examination in Progress -- Do Not Disturb," for any door leading to the examination area.
When Administering the AKT
When the time for administering the AKT has arrived and the preparations for testing have been made as previously described, the Proctor should proceed as follows:
1. Remove from the testing area any materials that might aid examinees in responding to questions on the examination.
2. Place a sign indicating that testing is in progress on each door that leads to the designated examination area ("Examination in Progress -- Do Not Disturb").
3. Check attendance, and be sure to verify the identification of all examinees sitting for the examination session.
4. If administering the AKT-1, make certain that those being pretested or post tested receive appropriate booklets (each is marked for pre or post administration on the cover).
5. Tell examinees not to open their test booklets until told by you to do so.
6. Distribute one examination booklet, one response sheet and one sharpened No. 2 pencil to each examinee.
7. Have each examinee complete the information blocks on the answer sheet as follows. Note that only No. 2 lead pencils may be used in making any entry on the response sheet.
Say to the examinees:
"Turn the response sheet sideways so that the information blocks are to the left. At the upper left, print your name in the blocks provided. Print your last name, then your first name, then your middle initial, if any. Do not use more space than is provided for each part of your name."
"Beneath each letter of your name, blacken with your pencil the corresponding letter on the grid."
If administering the AKT-1, pause until all have finished, then say:
"Just to the right of the name grid are two grids labeled 'TIME' and 'FORM'. In the block labeled 'TIME', blacken the circle labeled 'PRE' if you are taking the AKT-1 as a pretest, or blacken the circle labeled 'POST' if you are taking the AKT-1 as a post test. Check the front cover of your booklet to make sure it corresponds with your testing time."
"In the block labeled 'FORM', blacken the circle labeled 'A' if you are responding to Form A of the AKT-1, or blacken the circle labeled 'B' if you are responding to Form B. Check the front cover of your booklet to make sure that you have entered the proper form code."
Pause until all have finished, then say:
"Beneath the name grid are several grids that must be carefully completed. In the grid labeled 'Today's Date' blacken the circle that corresponds to the current month. In the blocks labeled 'Day' and 'Yr.', enter as digits the day of the current month and the last two digits of the current year."
"In each column, blacken the number beneath which corresponds to the number printed above."
Pause until all have finished, then say:
"In the blocks over the middle grid labeled 'Employee Identification Number', enter your Employee Identification Number. If you do not have an Employee Identification Number, leave this grid blank."
"Beneath each number entered in the Identification Number blocks, blacken the corresponding number in the grid."
Pause until all have finished, then say:
"In the next three blocks, labeled 'Institution Number', enter the number for your institution. It is the first three digits of the 'control number' that appears on the cover of your test booklet."
"Beneath the numbers you entered in the institution number blocks, blacken the corresponding numbers in the grid."
(PROCTOR NOTE: The block labeled "TYPE" should have a digit entered in it that describes the type of examinees to whom the AKT is being given. The codes are as follow:
Code '1' = A resident physician responding to the AKT-1 at the beginning or end of the first month of training.
Code '2' = A resident physician responding to the AKT-6 at about the sixth month of training.
Code ‘3’ = A resident physician responding to the AKT-24 at about the 18th month of training.
Determine the proper code for your examinees before proceeding.)
Pause until the previous entries have been made. Then, using the code just determined, say:
"In the block labeled 'TYPE', enter the number '1' (or '2', or ‘3’ if appropriate)."
"In the grid below, blacken the number entered above."
(PROCTOR NOTE: If you are administering either the AKT-1 or the AKT-6, have the examinees leave the block labeled 'X' blank. If you are administering the AKT-24, have the examinees complete the X block as follows.)
IF YOU ARE ADMINISTERING THE AKT-24:
“Now, open the cover of your test booklet and find the section labeled ‘Subspecialty Area Directions’ found at the top of the right hand page. First, read the opening paragraph, then select EACH additional subspecialty area for which you have had training and wish scores to be reported by placing a check mark in the box next to the subspecialty areas listed.”
Pause until all have finished, then say:
“Next, on the response sheet, find the block to the right of the ‘TYPE’ block labeled ‘X’. Beneath the heading ‘X’ are the numbers 1 through 9. Blacken EACH number in the ‘X’ block that was checked above. Leave entries 7-9 unmarked.”
“Please make certain that your selections are correct, and are properly blackened."
Pause until all have finished, then say:
"Notice that the response block for each item consists of 5 lettered circles, labeled 'A' through 'E', and that the item response areas are arranged in columns, each of which contains response areas for ten items. Responses to the first 80 items are to be recorded on the front of the response sheet. Responses to the remaining items should be made on the reverse side of the response sheet." (Have examinees turn over the response sheet to inspect the reverse side).
"When entering your response to each question, be sure to blacken the entire circle that is appropriate for your intended response. Do not check or 'X' out the circle. Examples of right and wrong ways to respond to an item are given toward the right side of the back of the answer sheet."
"Are there any questions concerning how you should record your responses to the test questions?"
Answer any questions that arise, then say:
"Now, open the cover of your test booklet and read the section labeled, 'General Directions', found on the right hand page. Ask any questions you may have."
"Note that it is to your advantage to respond to all questions even if you must guess."
"Next, complete the block on the inside of the front cover labeled, 'To Be Completed by Examinee'."
Pause until all have finished.
Answer any questions that the examinees ask concerning the administration of the instrument. Then, re-emphasize the importance of properly marking the answer sheet as shown in the specimen. Also, make certain that the examinees realize they will have two hours to complete the examination (two and one half hours if taking the AKT-24).
When all administrative problems have been resolved, tell the examinees to turn the first page and begin the examination.
After administering the AKT
1. When two hours of examination time have elapsed (two and one half hours if administering the AKT-24), tell the examinees to stop. Gather the test materials from the examinees and dismiss them.
2. In each test booklet, complete the block on the inside front cover labeled, "To Be Completed By Proctor." Note any irregularities and sign your name where indicated.
3. Check each response sheet to be certain that examinees have properly completed the identification blocks, have erased completely any entries which were to have been changed and have made no stray marks anywhere. Correct any recording errors noted.
4. Insert each examinee's response sheet in his test booklet and return the testing materials to the Institution Examiner. All materials should then be returned in the envelope provided to:
METRICS ASSOCIATES, INC.
13 Alpine Lane
Chelmsford, MA 01824