
                          GGGGGGGGGG   EEEEEEEEEEEE  NN        NN    OOOOOOOO    VV        VV   AAAAAAAAAA 
                         GGGGGGGGGGGG  EEEEEEEEEEEE  NNN       NN   OOOOOOOOOO   VV        VV  AAAAAAAAAAAA
                         GG         G  EE            NNNN      NN  OO        OO  VV        VV  AA        AA
                         GG            EE            NN NN     NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN  NN    NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN   NN   NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN    NN  NN  OO        OO    VV    VV    AA        AA
                         GG            EEEEEEEE      NN     NN NN  OO        OO    VV    VV    AA        AA
                         GG      GGGG  EEEEEEEE      NN      NNNN  OO        OO    VV    VV    AAAAAAAAAAAA
                         GG      GGGG  EE            NN       NNN  OO        OO     VV  VV     AAAAAAAAAAAA
                         GG        GG  EE            NN        NN  OO        OO     VV  VV     AA        AA
                         GG        GG  EE            NN        NN  OO        OO     VV  VV     AA        AA
                         GG        GG  EE            NN        NN  OO        OO      V  V      AA        AA
                         GG        GG  EE            NN        NN  OO        OO      VVVV      AA        AA
                         GGGGGGGGGGGG  EEEEEEEEEEEE  NN        NN   OOOOOOOOOO       VVVV      AA        AA
                          GGGGGGGGGG   EEEEEEEEEEEE  NN        NN    OOOOOOOO         VV       AA        AA


                                           A GENERAL PURPOSE ANALYSIS OF VARIANCE SYSTEM
                                             ---                      -  --

                                      GENOVA IS A FORTRAN 77 PROGRAM FOR ANALYSIS OF VARIANCE
                                        AND GENERALIZABILITY ANALYSES WITH BALANCED DESIGNS


                                                             AUTHORS

                                                         Joe E. Crick, Ed.D.
                                               Chief Technology & Information Officer
                                         Vice President Applications and Database Services
                                                National Board of Medical Examiners
                                                      Philadelphia, PA  19104

                                                      Robert L. Brennan, Ed.D.
                                                   Director, Iowa Testing Program
                                                         University of Iowa
                                                       Iowa City, Iowa 52242


                                                           VERSION 3.1
                                                          January, 2001
                               GENOVA has been checked for accuracy of output, however the authors
                                can make no assurances that the program is totally without error.

              GENOVA was developed in part under contract No. N00123-78-C-1206 with the Navy Personnel Research and 
              Development Center (NPRDC); Robert L. Brennan Principal Investigator.  GENOVA does not necessarily
              reflect NPRDC positions or policy, and no official endorsement should be inferred
   GENOVA  VERSION 3.1                                                                                                PAGE       1
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   STUDY       GSTUDY CONTROL CARD                                                 
   COMMENT     [p x i] design                                                      
   OPTIONS     RECORDS 2                                                           
   EFFECT      * P 36 0                                                            
   EFFECT      + I 30 0                                                            
   FORMAT      (30F2.0)                                                            
   PROCESS     7                                                                   
   GENOVA  VERSION 3.1                                                                                                PAGE       2
          G STUDY                                       GSTUDY CONTROL CARD                         

                                             EXPANDED MAIN AND INTERACTION EFFECT TABLE             

         (** = INFINITE)    P         I                  TOTAL   DEGREES                                                          
         SAMPLE SIZE       36        30        PRIMARY  NUMBER     OF                                                             
         UNIVERSE SIZE   ****      ****        INDICES  INDICES  FREEDOM                                                          
        ************************************
        *              *         *         *
        *  P           *    1    *    0    *      1        1         35                                                           
        *  I           *    0    *    1    *      1        1         29                                                           
        *              *         *         *
        ************************************
        *              *         *         *
        *  PI          *    1    *    1    *      2        2       1015                                                           
        *              *         *         *
        ************************************
   GENOVA  VERSION 3.1                                                                                                PAGE       3
          G STUDY                                       GSTUDY CONTROL CARD                         

                                               INPUT RECORD LISTING WITH RECORD MEANS               


 RECORD #      1     1.00000     1.00000     1.00000     0.00000     0.00000     1.00000     1.00000     1.00000
                     1.00000     0.00000     0.00000     0.00000     1.00000     1.00000     1.00000     1.00000
                     1.00000     1.00000     1.00000     1.00000     1.00000     1.00000     0.00000     0.00000
                     1.00000     0.00000     1.00000     0.00000     1.00000     1.00000     0.70000

 RECORD #      2     0.00000     0.00000     0.00000     1.00000     0.00000     0.00000     0.00000     0.00000
                     0.00000     0.00000     0.00000     0.00000     0.00000     0.00000     0.00000     0.00000
                     0.00000     1.00000     1.00000     1.00000     0.00000     0.00000     1.00000     1.00000
                     0.00000     0.00000     1.00000     0.00000     1.00000     1.00000     0.30000

 RECORD #     35     1.00000     1.00000     1.00000     0.00000     1.00000     1.00000     1.00000     1.00000
                     1.00000     0.00000     0.00000     1.00000     1.00000     0.00000     1.00000     1.00000
                     1.00000     1.00000     1.00000     1.00000     1.00000     0.00000     1.00000     0.00000
                     1.00000     0.00000     0.00000     1.00000     1.00000     1.00000     0.73333

 RECORD #     36     0.00000     0.00000     1.00000     0.00000     0.00000     0.00000     1.00000     1.00000
                     1.00000     0.00000     1.00000     0.00000     1.00000     1.00000     1.00000     1.00000
                     1.00000     0.00000     0.00000     1.00000     1.00000     1.00000     0.00000     1.00000
                     1.00000     0.00000     1.00000     0.00000     1.00000     1.00000     0.60000
   GENOVA  VERSION 3.1                                                                                                PAGE       4
          G STUDY                                       GSTUDY CONTROL CARD                         

                                                          CELL MEAN SCORES                          


 **********************************************************************************************************************************

                                             *** GRAND MEAN =            0.6129630 ***


 **********************************************************************************************************************************

   MEAN SCORES FOR EFFECT:  I              SUBSCRIPT NOTATION:  (I)                


         (1) =           0.833333        (2) =           0.777778        (3) =           0.555556        (4) =           0.333333 
         (5) =           0.500000        (6) =           0.805556        (7) =           0.638889        (8) =           0.694444 
         (9) =           0.722222       (10) =           0.333333       (11) =           0.611111       (12) =           0.611111 
        (13) =           0.777778       (14) =           0.638889       (15) =           0.750000       (16) =           0.416667 
        (17) =           0.777778       (18) =           0.833333       (19) =           0.472222       (20) =           0.694444 
        (21) =           0.666667       (22) =           0.527778       (23) =           0.361111       (24) =           0.277778 
        (25) =           0.833333       (26) =           0.361111       (27) =           0.583333       (28) =           0.250000 
        (29) =           0.972222       (30) =           0.777778                                                                 
   GENOVA  VERSION 3.1                                                                                                PAGE       5
          G STUDY                                       GSTUDY CONTROL CARD                         

                                                            ANOVA TABLE                             

         (** = INFINITE)    P         I                                                                                           
         SAMPLE SIZE       36        30                                                                                           
         UNIVERSE SIZE   ****      ****                                                                                           
        ------------------------------------------------------------------------------------------------------------------
                      DEGREES      SUMS OF        SUMS OF                             (QF = QUASI F RATIO)
                         OF      SQUARES FOR    SQUARES FOR       MEAN            F            F-TEST DEGREES OF FREEDOM
         EFFECT       FREEDOM    MEAN SCORES   SCORE EFFECTS     SQUARES      STATISTIC        NUMERATOR      DENOMINATOR
        ------------------------------------------------------------------------------------------------------------------
         P                 35      427.86667       22.08519       0.63101       3.27936              35            1015           
         I                 29      444.61111       38.82963       1.33895       6.95858              29            1015           
        ------------------------------------------------------------------------------------------------------------------
         PI              1015      662.00000      195.30370       0.19242                                                         
        ------------------------------------------------------------------------------------------------------------------
         MEAN                      405.78148
        ------------------------------------------------------------------------------------------------------------------
         TOTAL           1079                     256.21852
        ------------------------------------------------------------------------------------------------------------------
           NOTE: FOR GENERALIZABILITY ANALYSES, F-STATISTICS SHOULD BE IGNORED
   GENOVA  VERSION 3.1                                                                                                PAGE       6
          G STUDY                                       GSTUDY CONTROL CARD                         

                                                          G STUDY RESULTS                           

         (** = INFINITE)    P         I                                                                                           
         SAMPLE SIZE       36        30                                                                                           
         UNIVERSE SIZE   ****      ****                                               QFM = QUADRATIC FORM                        
        -----------------------------------------------------------------------------
                                   M O D E L   V A R I A N C E   C O M P O N E N T S
                      DEGREES      - - - - -   - - - - - - - -   - - - - - - - - - -
                         OF          USING            USING EMS          STANDARD
         EFFECT       FREEDOM      ALGORITHM          EQUATIONS           ERROR
        -----------------------------------------------------------------------------
         P                 35        0.0146196          0.0146196          0.0048985                                              
         I                 29        0.0318482          0.0318482          0.0094500                                              
        -----------------------------------------------------------------------------
         PI              1015        0.1924174          0.1924174          0.0085329                                              
        -----------------------------------------------------------------------------
           NOTE: THE "ALGORITHM" AND "EMS" ESTIMATED VARIANCE COMPONENTS WILL BE
                 IDENTICAL IF THERE ARE NO NEGATIVE ESTIMATES
   GENOVA  VERSION 3.1                                                                                                PAGE       7
          G STUDY                                       GSTUDY CONTROL CARD                         

                                                   EXPECTED MEAN SQUARE EQUATIONS                   

         (** = INFINITE)    P         I                                                                                           
         SAMPLE SIZE       36        30                                                                                           
         UNIVERSE SIZE   ****      ****                                                                                           



 EMS(P)           =  1.00*VC(PI)  +  30.00*VC(P)                                                                                   

 EMS(I)           =  1.00*VC(PI)  +  36.00*VC(I)                                                                                   

 EMS(PI)          =  1.00*VC(PI)                                                                                                   
   GENOVA  VERSION 3.1                                                                                                PAGE       8
          G STUDY                                       GSTUDY CONTROL CARD                         

                                 VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (V) 


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000001     0.0000893
 PI              -0.0000024    -0.0000020     0.0000728
   GENOVA  VERSION 3.1                                                                                                PAGE       9
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   COMMENT     D STUDY CONTROL CARD                                                
   DSTUDY      P X I DESIGN                                                        
   DEFFECT     $ P 36 / 0                                                          
   DEFFECT       I 5 10 15 20 30 40 50 / 0                                         
   ENDDSTUDY                                                                       
   GENOVA  VERSION 3.1                                                                                                PAGE      10
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-001                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :     5                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000        5     0.00637       0.00189      0.03185     1.0000        5     0.00637       0.00189   
 PI             0.19242     1.0000        5     0.03848       0.00171      0.19242     1.0000        5     0.03848       0.00171   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.05310   0.23044   0.00509
            LOWER CASE DELTA     0.03848   0.19617   0.00171          GENERALIZABILITY COEFFICIENT =  0.27531  ( 0.37989)
            UPPER CASE DELTA     0.04485   0.21179   0.00251                                   PHI =  0.24582  ( 0.32594)
                        MEAN     0.00784   0.08857
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      11
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-001                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000036
 PI              -0.0000005    -0.0000001     0.0000029
   GENOVA  VERSION 3.1                                                                                                PAGE      12
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-002                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    10                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       10     0.00318       0.00095      0.03185     1.0000       10     0.00318       0.00095   
 PI             0.19242     1.0000       10     0.01924       0.00085      0.19242     1.0000       10     0.01924       0.00085   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.03386   0.18401   0.00492
            LOWER CASE DELTA     0.01924   0.13871   0.00085          GENERALIZABILITY COEFFICIENT =  0.43175  ( 0.75979)
            UPPER CASE DELTA     0.02243   0.14976   0.00126                                   PHI =  0.39463  ( 0.65189)
                        MEAN     0.00413   0.06423
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      13
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-002                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000009
 PI              -0.0000002     0.0000000     0.0000007
   GENOVA  VERSION 3.1                                                                                                PAGE      14
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-003                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    15                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       15     0.00212       0.00063      0.03185     1.0000       15     0.00212       0.00063   
 PI             0.19242     1.0000       15     0.01283       0.00057      0.19242     1.0000       15     0.01283       0.00057   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.02745   0.16567   0.00490
            LOWER CASE DELTA     0.01283   0.11326   0.00057          GENERALIZABILITY COEFFICIENT =  0.53264  ( 1.13968)
            UPPER CASE DELTA     0.01495   0.12227   0.00084                                   PHI =  0.49440  ( 0.97783)
                        MEAN     0.00289   0.05372
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      15
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-003                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000004
 PI              -0.0000002     0.0000000     0.0000003
   GENOVA  VERSION 3.1                                                                                                PAGE      16
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-004                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    20                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       20     0.00159       0.00047      0.03185     1.0000       20     0.00159       0.00047   
 PI             0.19242     1.0000       20     0.00962       0.00043      0.19242     1.0000       20     0.00962       0.00043   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.02424   0.15569   0.00489
            LOWER CASE DELTA     0.00962   0.09809   0.00043          GENERALIZABILITY COEFFICIENT =  0.60311  ( 1.51957)
            UPPER CASE DELTA     0.01121   0.10589   0.00063                                   PHI =  0.56593  ( 1.30377)
                        MEAN     0.00227   0.04760
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      17
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-004                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000002
 PI              -0.0000001     0.0000000     0.0000002
   GENOVA  VERSION 3.1                                                                                                PAGE      18
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-005                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    30                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       30     0.00106       0.00032      0.03185     1.0000       30     0.00106       0.00032   
 PI             0.19242     1.0000       30     0.00641       0.00028      0.19242     1.0000       30     0.00641       0.00028   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.02103   0.14503   0.00489
            LOWER CASE DELTA     0.00641   0.08009   0.00028          GENERALIZABILITY COEFFICIENT =  0.69506  ( 2.27936)
            UPPER CASE DELTA     0.00748   0.08646   0.00042                                   PHI =  0.66167  ( 1.95566)
                        MEAN     0.00165   0.04057
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      19
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-005                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000001
 PI              -0.0000001     0.0000000     0.0000001
   GENOVA  VERSION 3.1                                                                                                PAGE      20
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-006                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    40                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       40     0.00080       0.00024      0.03185     1.0000       40     0.00080       0.00024   
 PI             0.19242     1.0000       40     0.00481       0.00021      0.19242     1.0000       40     0.00481       0.00021   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.01943   0.13939   0.00489
            LOWER CASE DELTA     0.00481   0.06936   0.00021          GENERALIZABILITY COEFFICIENT =  0.75242  ( 3.03914)
            UPPER CASE DELTA     0.00561   0.07488   0.00031                                   PHI =  0.72280  ( 2.60755)
                        MEAN     0.00134   0.03655
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      21
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-006                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000001
 PI              -0.0000001     0.0000000     0.0000000
   GENOVA  VERSION 3.1                                                                                                PAGE      22
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-007                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I      
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE                                                    
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE                                                    
         D STUDY SAMPLE SIZE :    36          D STUDY SAMPLE SIZES :    50                                                       
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.01462     1.0000        1     0.01462       0.00490      0.01462     1.0000        1     0.01462       0.00490   
 I              0.03185     1.0000       50     0.00064       0.00019      0.03185     1.0000       50     0.00064       0.00019   
 PI             0.19242     1.0000       50     0.00385       0.00017      0.19242     1.0000       50     0.00385       0.00017   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.01462   0.12091   0.00490
     EXPECTED OBSERVED SCORE     0.01847   0.13590   0.00489
            LOWER CASE DELTA     0.00385   0.06204   0.00017          GENERALIZABILITY COEFFICIENT =  0.79162  ( 3.79893)
            UPPER CASE DELTA     0.00449   0.06697   0.00025                                   PHI =  0.76523  ( 3.25944)
                        MEAN     0.00115   0.03391
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      23
          D STUDY                                           P X I DESIGN                            

                                                   D STUDY DESIGN NUMBER 001-007                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             PI                                                                                  

 P                0.0000240
 I                0.0000000     0.0000000
 PI               0.0000000     0.0000000     0.0000000
   GENOVA  VERSION 3.1                                                                                                PAGE      24
          D STUDY                                           P X I DESIGN                            



                              SUMMARY OF D STUDY RESULTS FOR SET OF CONTROL CARDS NO. 001

 ---------------------------------------------------------------------------------------------------------------------------------
                                                                          V A R I A N C E S
                            SAMPLE SIZES               --------------------------------------------------------
 D STUDY        -------------------------------------                 EXPECTED     LOWER     UPPER
  DESIGN  INDEX=    $P     I                             UNIVERSE     OBSERVED      CASE      CASE               GEN.
    NO    UNIV.=  INF.  INF.                               SCORE        SCORE      DELTA     DELTA     MEAN      COEF.      PHI
 ---------------------------------------------------------------------------------------------------------------------------------
 001-001            36     5                              0.01462      0.05310   0.03848   0.04485   0.00784   0.27531   0.24582
 001-002            36    10                              0.01462      0.03386   0.01924   0.02243   0.00413   0.43175   0.39463
 001-003            36    15                              0.01462      0.02745   0.01283   0.01495   0.00289   0.53264   0.49440
 001-004            36    20                              0.01462      0.02424   0.00962   0.01121   0.00227   0.60311   0.56593
 001-005            36    30                              0.01462      0.02103   0.00641   0.00748   0.00165   0.69506   0.66167
 001-006            36    40                              0.01462      0.01943   0.00481   0.00561   0.00134   0.75242   0.72280
 001-007            36    50                              0.01462      0.01847   0.00385   0.00449   0.00115   0.79162   0.76523
   GENOVA  VERSION 3.1                                                                                                PAGE      25
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   FINISH                                                                          
