
                          GGGGGGGGGG   EEEEEEEEEEEE  NN        NN    OOOOOOOO    VV        VV   AAAAAAAAAA 
                         GGGGGGGGGGGG  EEEEEEEEEEEE  NNN       NN   OOOOOOOOOO   VV        VV  AAAAAAAAAAAA
                         GG         G  EE            NNNN      NN  OO        OO  VV        VV  AA        AA
                         GG            EE            NN NN     NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN  NN    NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN   NN   NN  OO        OO   VV      VV   AA        AA
                         GG            EE            NN    NN  NN  OO        OO    VV    VV    AA        AA
                         GG            EEEEEEEE      NN     NN NN  OO        OO    VV    VV    AA        AA
                         GG      GGGG  EEEEEEEE      NN      NNNN  OO        OO    VV    VV    AAAAAAAAAAAA
                         GG      GGGG  EE            NN       NNN  OO        OO     VV  VV     AAAAAAAAAAAA
                         GG        GG  EE            NN        NN  OO        OO     VV  VV     AA        AA
                         GG        GG  EE            NN        NN  OO        OO     VV  VV     AA        AA
                         GG        GG  EE            NN        NN  OO        OO      V  V      AA        AA
                         GG        GG  EE            NN        NN  OO        OO      VVVV      AA        AA
                         GGGGGGGGGGGG  EEEEEEEEEEEE  NN        NN   OOOOOOOOOO       VVVV      AA        AA
                          GGGGGGGGGG   EEEEEEEEEEEE  NN        NN    OOOOOOOO         VV       AA        AA


                                           A GENERAL PURPOSE ANALYSIS OF VARIANCE SYSTEM
                                             ---                      -  --

                                      GENOVA IS A FORTRAN 77 PROGRAM FOR ANALYSIS OF VARIANCE
                                        AND GENERALIZABILITY ANALYSES WITH BALANCED DESIGNS


                                                             AUTHORS

                                                         Joe E. Crick, Ed.D.
                                               Chief Technology & Information Officer
                                         Vice President Applications and Database Services
                                                National Board of Medical Examiners
                                                      Philadelphia, PA  19104

                                                      Robert L. Brennan, Ed.D.
                                                   Director, Iowa Testing Program
                                                         University of Iowa
                                                       Iowa City, Iowa 52242


                                                           VERSION 3.1
                                                          January, 2001
                               GENOVA has been checked for accuracy of output, however the authors
                                can make no assurances that the program is totally without error.

              GENOVA was developed in part under contract No. N00123-78-C-1206 with the Navy Personnel Research and 
              Development Center (NPRDC); Robert L. Brennan Principal Investigator.  GENOVA does not necessarily
              reflect NPRDC positions or policy, and no official endorsement should be inferred
   GENOVA  VERSION 3.1                                                                                                PAGE       1
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   STUDY       P X (r:i) DESIGN                                                    
   COMMENT                                                                         
   OPTIONS     RECORDS 2                                                           
   EFFECT      * P 10 0                                                            
   EFFECT      + I 3 0                                                             
   EFFECT      + R:I 4 0                                                           
   FORMAT      (12F2.0)                                                            
   PROCESS     7                                                                   
   GENOVA  VERSION 3.1                                                                                                PAGE       2
          G STUDY                                         P X (r:i) DESIGN                          

                                             EXPANDED MAIN AND INTERACTION EFFECT TABLE             

         (** = INFINITE)    P         I         R                  TOTAL   DEGREES                                                
         SAMPLE SIZE       10         3         4        PRIMARY  NUMBER     OF                                                   
         UNIVERSE SIZE   ****      ****      ****        INDICES  INDICES  FREEDOM                                                
        **********************************************
        *              *         *         *         *
        *  P           *    1    *    0    *    0    *      1        1          9                                                 
        *  I           *    0    *    1    *    0    *      1        1          2                                                 
        *  R:I         *    0    *    2    *    1    *      1        2          9                                                 
        *              *         *         *         *
        **********************************************
        *              *         *         *         *
        *  PI          *    1    *    1    *    0    *      2        2         18                                                 
        *  PR:I        *    1    *    2    *    1    *      2        3         81                                                 
        *              *         *         *         *
        **********************************************
   GENOVA  VERSION 3.1                                                                                                PAGE       3
          G STUDY                                         P X (r:i) DESIGN                          

                                               INPUT RECORD LISTING WITH RECORD MEANS               


 RECORD #      1     5.00000     6.00000     5.00000     5.00000     5.00000     3.00000     4.00000     5.00000
                     6.00000     7.00000     3.00000     3.00000     4.75000

 RECORD #      2     9.00000     3.00000     7.00000     7.00000     7.00000     5.00000     5.00000     5.00000
                     7.00000     7.00000     5.00000     2.00000     5.75000

 RECORD #      9     9.00000     9.00000     8.00000     8.00000     6.00000     6.00000     6.00000     5.00000
                     5.00000     8.00000     1.00000     1.00000     6.00000

 RECORD #     10     4.00000     4.00000     4.00000     3.00000     3.00000     5.00000     6.00000     5.00000
                     5.00000     7.00000     1.00000     1.00000     4.00000
   GENOVA  VERSION 3.1                                                                                                PAGE       4
          G STUDY                                         P X (r:i) DESIGN                          

                                                          CELL MEAN SCORES                          


 **********************************************************************************************************************************

                                             *** GRAND MEAN =            4.7500000 ***


 **********************************************************************************************************************************

   MEAN SCORES FOR EFFECT:  I              SUBSCRIPT NOTATION:  (I)                


         (1) =           5.500000        (2) =           4.800000        (3) =           3.950000                                 

 **********************************************************************************************************************************

   MEAN SCORES FOR EFFECT:  R:I            SUBSCRIPT NOTATION:  (I,R)              


        (1,1) =          6.100000       (1,2) =          4.800000       (1,3) =          5.600000       (1,4) =          5.500000 
        (2,1) =          5.300000       (2,2) =          4.300000       (2,3) =          4.700000       (2,4) =          4.900000 
        (3,1) =          4.800000       (3,2) =          5.600000       (3,3) =          2.600000       (3,4) =          2.800000 
   GENOVA  VERSION 3.1                                                                                                PAGE       5
          G STUDY                                         P X (r:i) DESIGN                          

                                                            ANOVA TABLE                             

         (** = INFINITE)    P         I         R                                                                                 
         SAMPLE SIZE       10         3         4                                                                                 
         UNIVERSE SIZE   ****      ****      ****                                                                                 
        ------------------------------------------------------------------------------------------------------------------
                      DEGREES      SUMS OF        SUMS OF                             (QF = QUASI F RATIO)
                         OF      SQUARES FOR    SQUARES FOR       MEAN            F            F-TEST DEGREES OF FREEDOM
         EFFECT       FREEDOM    MEAN SCORES   SCORE EFFECTS     SQUARES      STATISTIC        NUMERATOR      DENOMINATOR
        ------------------------------------------------------------------------------------------------------------------
         P                  9     2800.16667       92.66667      10.29630       2.22935               9              18           
         I                  2     2755.70000       48.20000      24.10000       2.17238 QF            2 QF           12 QF        
         R:I                9     2835.40000       79.70000       8.85556       3.72044               9              81           
        ------------------------------------------------------------------------------------------------------------------
         PI                18     2931.50000       83.13333       4.61852       1.94035              18              81           
         PR:I              81     3204.00000      192.80000       2.38025                                                         
        ------------------------------------------------------------------------------------------------------------------
         MEAN                     2707.50000
        ------------------------------------------------------------------------------------------------------------------
         TOTAL            119                     496.50000
        ------------------------------------------------------------------------------------------------------------------
           NOTE: FOR GENERALIZABILITY ANALYSES, F-STATISTICS SHOULD BE IGNORED
   GENOVA  VERSION 3.1                                                                                                PAGE       6
          G STUDY                                         P X (r:i) DESIGN                          

                                                          G STUDY RESULTS                           

         (** = INFINITE)    P         I         R                                                                                 
         SAMPLE SIZE       10         3         4                                                                                 
         UNIVERSE SIZE   ****      ****      ****                                     QFM = QUADRATIC FORM                        
        -----------------------------------------------------------------------------
                                   M O D E L   V A R I A N C E   C O M P O N E N T S
                      DEGREES      - - - - -   - - - - - - - -   - - - - - - - - - -
                         OF          USING            USING EMS          STANDARD
         EFFECT       FREEDOM      ALGORITHM          EQUATIONS           ERROR
        -----------------------------------------------------------------------------
         P                  9        0.4731481          0.4731481          0.3855758                                              
         I                  2        0.3251543          0.3251543          0.4379875                                              
         R:I                9        0.6475309          0.6475309          0.3794056                                              
        -----------------------------------------------------------------------------
         PI                18        0.5595679          0.5595679          0.3766291                                              
         PR:I              81        2.3802469          2.3802469          0.3694860                                              
        -----------------------------------------------------------------------------
           NOTE: THE "ALGORITHM" AND "EMS" ESTIMATED VARIANCE COMPONENTS WILL BE
                 IDENTICAL IF THERE ARE NO NEGATIVE ESTIMATES
   GENOVA  VERSION 3.1                                                                                                PAGE       7
          G STUDY                                         P X (r:i) DESIGN                          

                                                   EXPECTED MEAN SQUARE EQUATIONS                   

         (** = INFINITE)    P         I         R                                                                                 
         SAMPLE SIZE       10         3         4                                                                                 
         UNIVERSE SIZE   ****      ****      ****                                                                                 



 EMS(P)           =  1.00*VC(PR:I)  +  4.00*VC(PI)  +  12.00*VC(P)                                                                 

 EMS(I)           =  1.00*VC(PR:I)  +  4.00*VC(PI)  +  10.00*VC(R:I)  +  40.00*VC(I)                                               

 EMS(R:I)         =  1.00*VC(PR:I)  +  10.00*VC(R:I)                                                                               

 EMS(PI)          =  1.00*VC(PR:I)  +  4.00*VC(PI)                                                                                 

 EMS(PR:I)        =  1.00*VC(PR:I)                                                                                                 
   GENOVA  VERSION 3.1                                                                                                PAGE       8
          G STUDY                                         P X (r:i) DESIGN                          

                                 VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (V) 


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0044439     0.1918331
 R:I              0.0000000    -0.0359871     0.1439486
 PI              -0.0444390    -0.0141849     0.0034130     0.1418495
 PR:I             0.0000000     0.0034130    -0.0136520    -0.0341300     0.1365199
   GENOVA  VERSION 3.1                                                                                                PAGE       9
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   COMMENT     D STUDY CONTROL CARDS                                               
   DSTUDY      P X (R:I)                                                           
   DEFFECT     $ P                                                                 
   DEFFECT     I 1 2 3 4 5 6                                                       
   DEFFECT     R:I 12 6 4 3 2 2                                                    
   ENDDSTUDY                                                                       
   GENOVA  VERSION 3.1                                                                                                PAGE      10
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-001                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     1          12                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        1     0.32515       0.43799      0.32515     1.0000        1     0.32515       0.43799   
 R:I            0.64753     1.0000       12     0.05396       0.03162      0.64753     1.0000       12     0.05396       0.03162   
 PI             0.55957     1.0000        1     0.55957       0.37663      0.55957     1.0000        1     0.55957       0.37663   
 PR:I           2.38025     1.0000       12     0.19835       0.03079      2.38025     1.0000       12     0.19835       0.03079   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     1.23107   1.10954   0.44373
            LOWER CASE DELTA     0.75792   0.87059   0.37028          GENERALIZABILITY COEFFICIENT =  0.38434  ( 0.62427)
            UPPER CASE DELTA     1.13704   1.06632   0.54454                                   PHI =  0.29385  ( 0.41612)
                        MEAN     0.50222   0.70868
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      11
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-001                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0044439     0.1918331
 R:I              0.0000000    -0.0029989     0.0009996
 PI              -0.0444390    -0.0141849     0.0002844     0.1418495
 PR:I             0.0000000     0.0002844    -0.0000948    -0.0028442     0.0009481
   GENOVA  VERSION 3.1                                                                                                PAGE      12
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-002                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     2           6                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        2     0.16258       0.21899      0.32515     1.0000        2     0.16258       0.21899   
 R:I            0.64753     1.0000       12     0.05396       0.03162      0.64753     1.0000       12     0.05396       0.03162   
 PI             0.55957     1.0000        2     0.27978       0.18831      0.55957     1.0000        2     0.27978       0.18831   
 PR:I           2.38025     1.0000       12     0.19835       0.03079      2.38025     1.0000       12     0.19835       0.03079   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     0.95129   0.97534   0.37121
            LOWER CASE DELTA     0.47814   0.69148   0.18321          GENERALIZABILITY COEFFICIENT =  0.49738  ( 0.98956)
            UPPER CASE DELTA     0.69468   0.83347   0.26984                                   PHI =  0.40515  ( 0.68111)
                        MEAN     0.31167   0.55827
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      13
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-002                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0022219     0.0479583
 R:I              0.0000000    -0.0014995     0.0009996
 PI              -0.0222195    -0.0035462     0.0001422     0.0354624
 PR:I             0.0000000     0.0001422    -0.0000948    -0.0014221     0.0009481
   GENOVA  VERSION 3.1                                                                                                PAGE      14
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-003                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     3           4                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        3     0.10838       0.14600      0.32515     1.0000        3     0.10838       0.14600   
 R:I            0.64753     1.0000       12     0.05396       0.03162      0.64753     1.0000       12     0.05396       0.03162   
 PI             0.55957     1.0000        3     0.18652       0.12554      0.55957     1.0000        3     0.18652       0.12554   
 PR:I           2.38025     1.0000       12     0.19835       0.03079      2.38025     1.0000       12     0.19835       0.03079   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     0.85802   0.92630   0.36586
            LOWER CASE DELTA     0.38488   0.62038   0.12171          GENERALIZABILITY COEFFICIENT =  0.55144  ( 1.22935)
            UPPER CASE DELTA     0.54722   0.73974   0.17935                                   PHI =  0.46370  ( 0.86464)
                        MEAN     0.24815   0.49814
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      15
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-003                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0014813     0.0213148
 R:I              0.0000000    -0.0009996     0.0009996
 PI              -0.0148130    -0.0015761     0.0000948     0.0157611
 PR:I             0.0000000     0.0000948    -0.0000948    -0.0009481     0.0009481
   GENOVA  VERSION 3.1                                                                                                PAGE      16
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-004                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     4           3                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        4     0.08129       0.10950      0.32515     1.0000        4     0.08129       0.10950   
 R:I            0.64753     1.0000       12     0.05396       0.03162      0.64753     1.0000       12     0.05396       0.03162   
 PI             0.55957     1.0000        4     0.13989       0.09416      0.55957     1.0000        4     0.13989       0.09416   
 PR:I           2.38025     1.0000       12     0.19835       0.03079      2.38025     1.0000       12     0.19835       0.03079   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     0.81139   0.90077   0.36721
            LOWER CASE DELTA     0.33825   0.58159   0.09161          GENERALIZABILITY COEFFICIENT =  0.58313  ( 1.39883)
            UPPER CASE DELTA     0.47350   0.68811   0.13492                                   PHI =  0.49982  ( 0.99927)
                        MEAN     0.21639   0.46518
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      17
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-004                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0011110     0.0119896
 R:I              0.0000000    -0.0007497     0.0009996
 PI              -0.0111097    -0.0008866     0.0000711     0.0088656
 PR:I             0.0000000     0.0000711    -0.0000948    -0.0007110     0.0009481
   GENOVA  VERSION 3.1                                                                                                PAGE      18
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-005                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     5           2                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        5     0.06503       0.08760      0.32515     1.0000        5     0.06503       0.08760   
 R:I            0.64753     1.0000       10     0.06475       0.03794      0.64753     1.0000       10     0.06475       0.03794   
 PI             0.55957     1.0000        5     0.11191       0.07533      0.55957     1.0000        5     0.11191       0.07533   
 PR:I           2.38025     1.0000       10     0.23802       0.03695      2.38025     1.0000       10     0.23802       0.03695   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     0.82309   0.90724   0.36955
            LOWER CASE DELTA     0.34994   0.59156   0.07533          GENERALIZABILITY COEFFICIENT =  0.57485  ( 1.35209)
            UPPER CASE DELTA     0.47972   0.69262   0.11051                                   PHI =  0.49655  ( 0.98630)
                        MEAN     0.21209   0.46054
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      19
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-005                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0008888     0.0076733
 R:I              0.0000000    -0.0007197     0.0014395
 PI              -0.0088878    -0.0005674     0.0000683     0.0056740
 PR:I             0.0000000     0.0000683    -0.0001365    -0.0006826     0.0013652
   GENOVA  VERSION 3.1                                                                                                PAGE      20
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-006                    

       OBJECT OF MEASUREMENT :      P                       FACETS :      I          R:I     
     G STUDY POPULATION SIZE : INFINITE     G STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
     D STUDY POPULATION SIZE : INFINITE     D STUDY UNIVERSE SIZES : INFINITE    INFINITE                                        
         D STUDY SAMPLE SIZE :    10          D STUDY SAMPLE SIZES :     6           2                                           
 ----------------------------------------------------------------------------------------------------------------------------------
                          VARIANCE COMPONENTS IN TERMS OF                            VARIANCE COMPONENTS IN TERMS OF
                G STUDY UNIVERSE (OF ADMISSIBLE OBSERVATIONS) SIZES             D STUDY UNIVERSE (OF GENERALIZATION) SIZES
             ---------------------------------------------------------- -----------------------------------------------------------
                                                VARIANCE COMPONENTS                                        VARIANCE COMPONENTS
               VARIANCE     FINITE   D STUDY      FOR MEAN SCORES         VARIANCE     FINITE   D STUDY      FOR MEAN SCORES
              COMPONENTS   UNIVERSE SAMPLING --------------------------  COMPONENTS   UNIVERSE SAMPLING ---------------------------
              FOR SINGLE     COR-     FRE-                   STANDARD    FOR SINGLE     COR-     FRE-                   STANDARD
 EFFECT      OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS    OBSERVATIONS  RECTIONS QUENCIES   ESTIMATES      ERRORS
 ----------------------------------------------------------------------------------------------------------------------------------
 P              0.47315     1.0000        1     0.47315       0.38558      0.47315     1.0000        1     0.47315       0.38558   
 I              0.32515     1.0000        6     0.05419       0.07300      0.32515     1.0000        6     0.05419       0.07300   
 R:I            0.64753     1.0000       12     0.05396       0.03162      0.64753     1.0000       12     0.05396       0.03162   
 PI             0.55957     1.0000        6     0.09326       0.06277      0.55957     1.0000        6     0.09326       0.06277   
 PR:I           2.38025     1.0000       12     0.19835       0.03079      2.38025     1.0000       12     0.19835       0.03079   
 ----------------------------------------------------------------------------------------------------------------------------------
            QFM = QUADRATIC FORM

 ----------------------------------------------------------------------------------------------------------------------------------


                                                    STANDARD
                                          STANDARD  ERROR OF
                                VARIANCE DEVIATION  VARIANCE

              UNIVERSE SCORE     0.47315   0.68786   0.38558
     EXPECTED OBSERVED SCORE     0.76476   0.87451   0.37121
            LOWER CASE DELTA     0.29162   0.54001   0.06277          GENERALIZABILITY COEFFICIENT =  0.61869  ( 1.62251)
            UPPER CASE DELTA     0.39977   0.63227   0.09209                                   PHI =  0.54203  ( 1.18356)
                        MEAN     0.18463   0.42969
 ----------------------------------------------------------------------------------------------------------------------------------
        NOTE:  SIGNAL/NOISE RATIOS ARE IN PARENTHESES
   GENOVA  VERSION 3.1                                                                                                PAGE      21
          D STUDY                                            P X (R:I)                              

                                                   D STUDY DESIGN NUMBER 001-006                    


         VARIANCE - COVARIANCE MATRIX FOR ESTIMATED VARIANCE COMPONENTS (FOR MEAN SCORES) IN UNIVERSE OF GENERALIZATION (W)


                  P             I             R:I           PI            PR:I                                                    

 P                0.1486687
 I                0.0007406     0.0053287
 R:I              0.0000000    -0.0004998     0.0009996
 PI              -0.0074065    -0.0003940     0.0000474     0.0039403
 PR:I             0.0000000     0.0000474    -0.0000948    -0.0004740     0.0009481
   GENOVA  VERSION 3.1                                                                                                PAGE      22
          D STUDY                                            P X (R:I)                              



                              SUMMARY OF D STUDY RESULTS FOR SET OF CONTROL CARDS NO. 001

 ---------------------------------------------------------------------------------------------------------------------------------
                                                                          V A R I A N C E S
                            SAMPLE SIZES               --------------------------------------------------------
 D STUDY        -------------------------------------                 EXPECTED     LOWER     UPPER
  DESIGN  INDEX=    $P     I     R                       UNIVERSE     OBSERVED      CASE      CASE               GEN.
    NO    UNIV.=  INF.  INF.  INF.                         SCORE        SCORE      DELTA     DELTA     MEAN      COEF.      PHI
 ---------------------------------------------------------------------------------------------------------------------------------
 001-001            10     1    12                        0.47315      1.23107   0.75792   1.13704   0.50222   0.38434   0.29385
 001-002            10     2     6                        0.47315      0.95129   0.47814   0.69468   0.31167   0.49738   0.40515
 001-003            10     3     4                        0.47315      0.85802   0.38488   0.54722   0.24815   0.55144   0.46370
 001-004            10     4     3                        0.47315      0.81139   0.33825   0.47350   0.21639   0.58313   0.49982
 001-005            10     5     2                        0.47315      0.82309   0.34994   0.47972   0.21209   0.57485   0.49655
 001-006            10     6     2                        0.47315      0.76476   0.29162   0.39977   0.18463   0.61869   0.54203
   GENOVA  VERSION 3.1                                                                                                PAGE      23
                                                     CONTROL CARD INPUT LISTING


   COLUMN   11111111112222222222333333333344444444445555555555666666666677777777778
   12345678901234567890123456789012345678901234567890123456789012345678901234567890
   FINISH                                                                          
