April 2000 Board of Directors Meeting Minutes (draft)
New Orleans, Louisiana
April 24, 2000
Announcements and Review of Agenda
President Kolen called the meeting to order, reviewed the agenda, and asked if there were any modifications or additions to the agenda. Dr. Kolen noted that he intended to continue the practice of asking for a moment of silence during the Breakfast meeting for recently departed NCME members such as Frederick Lord.
Approval of the Minutes
The draft minutes of the November 21, 1999 Board of Directors Meeting were approved without amendment.
Future Revisions of the Test Standards
An ad hoc committee was created to review and evaluate the processes used in the development of the 1999 version of the Standards for Educational and Psychological Testing. The Committee's report offered 14 recommendations designed to improve the procedures used to develop the Standards and to address some of the concerns expressed by Board members during the development process. The Committee's report was discussed by the Board. The central recommendation, requiring a bylaw change, would ensure that individuals who work for testing organizations not be excluded from serving on the next Standards revision committee.
The Board felt that the rationale for excluding members who work at testing organizations from serving on the revision committee as not compelling. The appearance of a conflict of interest was not appropriate given academics with extensive consulting work with testing organizations or individuals from research organizations that are involved with test construction are included. The Committee felt that issues associated with the usefulness of the Standards could be traced to the exclusion from the committee of individuals whose primary places of employment are testing organizations. The Committee felt that the disenfranchisement of a large segment of the NCME membership was serious enough to recommend the following bylaw change: "Invitation to participate in NCME-sponsored committees will be made based on the requisite knowledge and expertise; other individual characteristics, including type of workplace, are to be considered only to ensure adequate representation of groups ofindividuals on committees." After discussion, the Board adopted the proposed bylaw change as a Board policy, rather than as a bylaw change that would be voted upon by the membership. They also indicated that the policy be limited to Joint Committee on Standards for Educational and Psychological Testing. The Board approved the other Committee recommendations concerning the role of sponsoring organizations and the process for developing and reviewing drafts of the Standards. Dr. Kolen was asked to use the committee's recommendations as discussion points with APA and AERA to formalize a new management agreement for the revision of the Standards for Educational and Psychological Testing. Discussion concluded with an expression of extreme gratitude on behalf of the field for the arduous work of the Standards Committee. The ad hoc committees recommendations were only concerned with process issues to govern the next revision.
The Board acknowledge with appreciation the excellent work of the President, Patricia McAllister, and the NCME Educational Testing Legislation and Policy Committee for responding to the Office of Civil Rights guidelines on the Use of Tests When Making High-Stakes Decisions for Students.
Joint Committee on Testing Practices
Dr. Kolen noted that JCTP had begun the process to revised the Code of Fair Testing Practices in Education, which requires revision in light of the 1999 Standards. Dr. Nobel, one of NCME's JCTP representatives and JCTP co-chair, stated that the intent of the revision was not to reconceptualize or redesign the Code, but to make changes as required by the new Standards. The Board will have the opportunity to comment on drafts and the right to endorse the final product. It was noted that the Board would ask its Standards and Test Use Committee to review drafts of the Code.
Dr. Nobel reported that next fall a working group would likely be proposed to develop a casebook on the Test Standards as an educational tool. Also, JCTP's bylaws were being reviewed for possible revisions in the areas of the selection of members for JCTP working groups and the endorsement process of JCTP products.
The National Association of Test Directors (NATD) requested to be a member of JCTP. Their request for membership must be approved by all the current sponsors of JCTP. The Board approved the President's supporting the request when the formal letter is receive by NCME.
Annual Meeting
The 2000 Annual Meeting Program Committee recommended that 1) maximum participation in the Program be related to submissions versus acceptance, 2) two discussants be assigned to sessions if the format were appropriate, and 3) the practice of sending postcards to all members soliciting their involvement be continued. The discussion of the report began with kudos to Dr. Harris for her work in developing an excellent Program. The report also offered several suggestions that the 2001 Program Committee might want to consider. For example, only one paper was received in the area of policy. Thus, the call for reviewers in this area might be deleted next year given the 82 members who volunteered to review submissions on the topic. The Board felt that the notion of having two discussants for appropriate sessions was desirable and suggested that the intention be highlighted in the Call for Papers.
In reviewing the Professional Development and Training program held during the Annual Meeting, the Board suggested that the Committee focus next year on expanding the program to provide training opportunities at times other than at the Annual Meeting.
Publications
Dr. Frisbie was pleased to report that the NCME Website was fully functional. A report was distributed that detailed the contents of the website (e.g. Annual Meeting Program, Board Minutes, electronic version of the Newsletter) and tracked the increasing number of hits since it became operational. The Committee encourages committee chairs to consider how the website could be used to enhance its work. The term of the Website editor was briefly discussed. When Dr. Harnisch completes his term NCME might want to consider contracting with a commercial firm to maintain the website in order to divorce it from the editorial function as condition of the next editorial appointment. Coupled with the consideration of a new editor and a commercial vendor for the website, should be a consideration of the option of electronic submissions to the Annual Meeting Program and having accepted papers available on the Website that are linked to the Program.
The Publications Committee suggested consideration of putting EM:IP on line. Was there a compelling reason to do so rather than publishing only a table of contents and abstracts of articles? The question of possible loss of subscription sales was raised. The matter was referred back to the Committee for further consideration and analysis.
It was noted thatthe International Measurement Issues Committee requested that an internationalcolumn become a permanent part of EM:IP. The practice started two years ago and had reached the point where the section was full for the remainder of current editor's term. Dr. Benson, incoming EM:IP editor, has indicated her willingness to continue the column. The questions raised were, should manuscripts be sent to a special editor, as was the case the past two years, and do the pages published come out of the existing page allocation of the journal? The sense of the Board was that continuation of the international column should not be made permanent, but should remain at the discretion of the editor. No other action was taken by the Board.
Dr. Frisbie called attention to the editors' reports that were included in the agenda package and noted that he had nothing to add to the information provided.It was agreed that the editor of the Newsletter should alter the publication schedule to more closely fit with activities before and after the Annual Meeting.
The Publications Committee, consistent with policy, provided a slate of candidates for the JEM Editorship to President Kolen. All the candidates had expressed a willingness to be nominated. Dr. Kolen presented a rank-ordered list of the candidates to the Board for review and approval. The Board endorsed the slate as presented. Dr. Kolen indicated that he would follow-up with the extending an invitation to edit JEM.
Adjournment
Dr. Kolen thanked Board members for their thoughtful participation and a motion to adjourn was approved.
Respectfully submitted,
William J. Russell
Executive Director
New Orleans, Louisiana
April 27, 2000
Announcements and Review of Agenda
President Fremer called the meeting to order, reviewed the agenda, and asked if there were any modifications or additions to the agenda. Dr. Fremer stated that he hoped that during his presidency NCME would pay special attention to promoting the public understanding of testing.
Joint Committee on Standards for Educational Evaluation
Dr. Yarborough, NCME's representative to the Joint Committee, briefly reviewed the work of the Committee since its inception in 1972. It has issued three sets of standards that are ANSI approved but not endorsed by the sponsoring organizations. The agenda package recalled the Board's discussion and concern about the Committee's proposal to develop student evaluation standards. Dr. Yarborough noted that the draft standards are available for review and comment on the Joint Committee's Website. He also noted that the session on the Annual Meeting Program on the draft standards was positively received. The concerns expressed were the lack of supporting material an questions about the practical value of the standards. Dr. Yarborough noted that during the next year the Committee would revise and rewrite the standards and submit them to field tests and hearings. Given the size of the document (350 pages) Board members questioned the utility of the Standards and means by which the standards would reach the intended audience. Dr. Yarborough reported that the Committee had not addressed the questions of translation or distribution. Finally, he noted that when the standards are endorsed by the Joint Committee, the inside cover of the document would list the organizational sponsors and include a statement that formal endorsement of standards was not sought or given by the sponsoring organizations. The Board requested that Dr. Yarborough seek the review and comment of the document by NCME's Standards and Test Use Committee.
Recruitment of Educational Measurement Specialists
The Committee reported that it had developed text for a career brochure that will be jointly published by NCME and Division 5 of APA. It had also revised and posted on the Website a list of internships in the field. The Committee has nearly completed updating the roster of graduate programs in educational measurement that will be posted on the Website. Activities for the next year include work on developing a Minority Undergraduate Students of Excellence Program and developing short biographies of various measurement professionals.
Dr. Kolen reported on the meeting he organized during the Annual Meeting to brainstorm recruitment strategies that have been successfully used by Measurement Programs to attract students. He thought the meeting was useful and generated a number of ideas. Soliciting contributions from members and institutions to sponsor a fellowship program and developing a video career brochure were some of the suggestions offered. There was a consensus that the field must do a much better job in marketing and promoting itself. He felt a similar meeting should be organized next year during the Annual Meeting to focus more on what NCME as an organization might do in the areas of recruitment and PR.
Committees
It was noted that some of the work of NCME committees was not being done completely or involving the active participation of all members of the committee. The notion of simply increasing the size of committees was not supported as it might make it more difficult for the chair to work with the committee. It was suggested, however, that a committee could expand in size on a temporary basis if the work load justified a larger committee. NCME should continue to build the database of members who have served NCME in various capacities. Dr. Fremer was pleased to support the notion that Area Directors should write committee members thanking them in advance for their service to NCME during the forthcoming year.
To a large degree, the functioning of a committee is correlated to the work of the Committee chair. They may need more careful monitoring and occasional prodding by the Area Director. Some committees like the award committees and nominations committee appear to be on automatic pilot. It might be beneficial to provide Committee chairs with suggestions of what to do. It might also be helpful for Area Directors to check in with their committee chairs during the summer.
2001 Annual Meeting
Drs. Schmidt and Potenza, cochairs of the 2001 Annual Meeting, shared their preliminary thoughts about the Seattle Annual Meeting. They solicited Board members' ideas for themes, topics, and speakers for the meeting. They intended to reinitiate the keynote address during the meeting. They would also think about how to generate media attention for the meeting. Dr. Potenza asked if they might be interested in organizing a site visit to Microsoft's "House of the Future." The Board reacted positively to the idea.
Software Issues in Educational Measurement
Dr. Hanson highlighted his ad hoc Committee's report that addressed what NCME might do to facilitate the availability of information about specialized software used in educational measurement and the dissemination of such software. He indicated that the report was a draft and that he intended to have a final report by the fall meeting of the Board. Of the several specific items listed by the committee as part of its charge, the Board strongly resonated to making recommendations about publications that NCME might produce on software documentation and dissemination, such as a position paper or guidelines. Interest was also expressed about the possibility of establishing a software exchange on the NCME website. Further discussion noted the complexity of the issues such as software documentation and levels of software dissemination. Discussion concluded (when H.D. nodded off) with a request that the item be placed on the agenda for the summer Board meeting as part of long-range planning. It was suggested that it was critical for NCME to get into this area early. Dr. Hanson was thanked for a tremendous job in initiating the discussion and consideration of an increasingly important topic.
Graduate Student Issues Committee
The Graduate student Issues Committee report was distributed at the meeting. The report was an information item that highlighted a very productive year for the Committee. The Committee was highly commended for its diligent service to graduate students and to the Council.
International Measurement Issues Committee
The International Committee's report requested that an international section of EM:IP be authorized as a permanent part of the journal was previously discussed as part of the Publications Committee report. The Board reiterated its support of the idea, but felt it should be left to the discretion of the editor of EM:IP. Dr. Benson, incoming editor, will have the opportunity to appoint an editor for the international section is she wishes it to be continued. It was noted the Committee would provide Dr. Benson with nominations for the position.
Budget and Auditor
The Board approved the President?s appointment of the CPA firm of Kirwan and Company to audit the Council's 2000 financial records.
The draft of the FY 2001 budget was briefly reviewed. After questions, the draft was approved as an operating budget until the fall meeting of the Board. The statement projected deficit budgets of $69,955 and $43,123 in fiscal years 2000 and 2001 respectively. The decline in the membership base was noted. It was requested that historical data on the size of membership be available for the summer Board meeting.
Next Meetings
Board members decided to hold the summer Board meeting in Iowa City on the dates of August 26-27, 2000. The fall meeting of the Board will held on December 2-3, 2000 at a Chicago airport hotel.
Agenda items suggested for the summer Board meeting included long range planning, adjustment to the committee structure, oral history project, membership recruitment.
Adjournment
Dr. Fremer thanked Board members for their kind participation and a motion to adjourn was approved.
Respectfully submitted,
William J. Russell
Executive Director