ED Seal

UNITED STATES DEPARTMENT OF EDUCATION
OFFICE OF ADMINISTRATIVE LAW JUDGES



APPLICATION OF THE
MICCOSUKEE CORPORATION,
Applicant.

Docket No. 97-9-O
Indian Vocational Education Program Proceeding

DECISION

Appearances: Michael Salem, Project Director, Miami, Florida, for the Miccosukee Corporation.

Daphna Crotty, Esq., of the Office of the General Counsel, United States Department of Education, Washington, D.C., for the Assistant Secretary for Vocational and Adult Education.

Before: Allan C. Lewis, Chief Administrative Law Judge

The Miccosukee Corporation (Miccosukee) seeks review of a decision by the Secretary not to include Miccosukee among the initial 25 applicants selected in a competitive process for an award of a grant under the Indian Vocational Education Program. 34 C.F.R. § 401.23. Miccosukee asserts a number of objections based on the procedures employed by the United States Department of Education (Department) in arriving at its score and a number of objections based on the actual scores assigned to various criteria by the readers. As a result of these objections, it appears that Miccosukee seeks to void the proposed grant awards based upon the procedures utilized by the Department, or, in the alternative, an increase in its score to place it within the group of successful applicants for the award of a grant. Based on findings and analysis set forth infra, the review process was proper and no adjustment in the score assigned to Miccosukee is warranted.

I. OPINION

The Indian Vocational Education Program provides financial assistance to projects that provide vocational education for the benefit of Native Americans. 34 C.F.R. § 40l.1. The Secretary provides financial assistance through grants, contracts, or cooperative agreements to plan, conduct, and administer projects that are authorized by and consistent with the purposes of the Carl D. Perkins Vocational and Applied Technology Education Act, Pub. L. No. 88-210, 77 Stat. 403 (1963), as amended by the Carl D. Perkins Vocational and Applied Technology Education Act Amendments of 1990, Pub. L. No. 101-392, 104 Stat. 753 (1990), and the National Dropout Prevention Act of 1991, Pub. L. No. 102-103, § 311, 105 Stat. 497, 505 (1991) (to be codified at 20 U.S.C. § 2301 et seq.).

Pursuant to this authorization, the Secretary published a Notice Inviting Applications for New Awards for Fiscal Year 1996 under the Indian Vocational Education Program on June 19, 1996. 61 Fed. Reg. 31,364. In this Notice, the Secretary invited eligible tribal organizations to submit applications to be evaluated using the selection criteria and corresponding weights contained in 34 C.F.R. § 401.21, as supplemented by 34 C.F.R. § 401.20(b) and 34 C.F.R. § 401.20(e). As a result of this Notice, 83 applications were received for fiscal year 1996. The Department deemed three applications ineligible, and, as such, 80 applications were evaluated for grant funding. The Department selected the top 25 eligible applications for grant funding with the lowest ranking successful applicant to be selected for funding receiving a standardized score of 82.l5.See footnote 1 1 Miccosukee received a standardized score of 44.48 and was ranked 67th of the 80 applications evaluated.See footnote 2 2

As part of the evaluation process, reviewers were provided with a Technical Review Form which contained written instructions regarding the criteria for evaluating the applications. The instruction section of the Technical Review Form provided a series of five suggested review procedures.See footnote 3 3 Miccosukee does not object to suggested review procedures 1, 2, and 5. Miccosukee does, however, challenge the third suggested review procedure -- "write your evaluation of the strengths and weaknesses of the application for each criterion." Specifically, Miccosukee alleges that this procedure is inadequate to assess the true strengths of the application.

The Department asserts that the strength of the application is determined by. the substance of the application and not the procedures employed for assessing the strengths and weaknesses of the application.

While the Department is correct, its argument fails to address whether the procedure, as utilized, is adequate. In a competitive grant process, each applicant elects to participate and compete in a process in which standards and evaluation criteria are disclosed prior to the submission of the grant application. As part of the review process, reviewers evaluate the application for its strengths and weaknesses which are used, in part, to justify the numerical rating assigned by the reviewer to a particular criterion. Moreover, such a process also serves to provide guidance to the applicant for future applications. Inasmuch as the process utilized herein is reasonable and an applicant can always elect not to compete for funding if it disagrees with the standards and criteria utilized, Miccosukee's initial argument is rejected.

Miccosukee also argues that, under the same suggested review procedure, some weaknesses identified by the reviewers are merely opinions containing their subjective views regarding the application. Accordingly, Miccosukee asserts that the reviewers considered extraneous material when they should have considered the established criteria exclusively. As such, Miccosukee concludes that process should be invalidated.

In this regard, the Department argues that the tribunal recognized in In re Sisseton Wahpeton Community College, Dkt. No. 95-86-0, U.S. Dept. of Education (Aug. 7, 1995) at 6, that "reviewers are required to render judgments in the evaluative process." Accordingly, the Department concludes that the subjective views of the review panel are an integral part of evaluating an application.

Miccosukee's argument appears to address the merits of the score assigned by the reviewers and not the procedures employed by the Department in this grant process. Nonetheless, Miccosukee's concern is vague and ambiguous inasmuch as it does not articulate specific instances in which the reviewers purportedly used inappropriate, subjective comments. Accordingly, such an objection is meaningless and is, therefore, rejected.

Miccosukee also argues that the rating criteria have no correlation to the success of the program and, therefore, should not be employed. As such, Miccosukee challenges the criteria selected by the Secretary to determine successful applicants.

While the Department does not appear to specifically address this objection, it is evident that, in publishing the notice inviting applications for the grant program, the Secretary selected the criteria which he believed were necessary to ultimately produce a successful program. The Secretary, in announcing the grants, possesses the sole discretion to establish the criteria to evaluate the applications. Given this discretion, the criteria will not be disturbed.

In addition, in challenging the criteria established by regulation, Miccosukee essentially challenges the validity of 34 C.F.R. § 401.21. As a threshold matter, the question arises whether an Administrative Law Judge has the authority to invalidate a regulation under the Administrative Procedure Act.See footnote 4 4 In this circumstance, it is clear that the tribunal has no authority to declare a regulation invalid. See Sisseton Wahpeton Community College, supra; In re Smithville R-II School District, Dkt. No. 91-4-I, U.S. Dept. of Education (Sec. Dec. July 27, 1992), In re Lemont Township High School District #210, Dkt. No. 89-48-I, U.S. Dept. of Education (Fete 6, 1992), certified by the Secretary (May 4, 1992). In light of the above, Miccosukee's argument is meritless.

Finally, Miccosukee argues that the procedures employed by the Department do not include an objective method of evaluation. In Miccosukee's view, successful grant applications were based on the subjective interpretations of reviewers. As such, Miccosukee argues that the objective criteria should define with greater specificity the elements necessary for a successful grant application. The Department counters that objective criteria are not required within this process.

Initially, the tribunal disagrees with the Department's assessment of the grant award process. The regulations under Part 401 of 34 C.F.R. establish objective criteria for the evaluation of applications. Moreover, the breadth of the criteria provide adequate guidance regarding elements to be considered in determining a numerical score based on a particular criterion. While the criteria are objective in nature, such criteria may require subjective evaluation. As such, there are adequate objective standards to guide the subjective evaluations thereunder.

Miccosukee also objects to the fourth suggested procedure outlined by the Department -- "upon completing the written review of each criterion, circle the descriptor on the scale and assign a score that best reflects your rating of the application with respect to that criterion."

In evaluating an application, a reviewer evaluates an application identifying its strengths and weaknesses. The reviewer, through a two-step process designed to reach a numerical score for each criterion, determines whether the application with respect to a particular criterion, falls into one of four categories -- (1) not addressed, (2) inadequate, (3) adequate, and (4) good to excellent. Each category provides a numerical range of scores depending on the number of points assigned to a particular criterion. For each criterion, the reviewer follows the same process and, after evaluating the application in light of all of the criteria, reaches a total score for the application.

Miccosukee asserts that the procedure does not provide an adequate justification for the numerical score assigned by a reviewer to a specific criterion. In this regard, it appears that Miccosukee deems the process inadequate to determine how the reviewer reached a score for each criterion. In Miccosukee's view, the criteria should establish, with more certainty, a manner in which to calculate an applicant's score.

Miccosukee's argument is similar to its argument above--that grant applications should be evaluated by more objective criteria. That argument was rejected and need not be discussed further.

Miccosukee also raises a number of objections with respect to the score assigned to its application. In this regard, Miccosukee raises a number of arguments under each of the criterion established to evaluate the grant applications. According to Miccosukee, the reviewers overlooked, misunderstood, or misinterpreted a number of aspects of its grant application which, in its view, would have modified its score with respect to a number of criteria. As such, Miccosukee's arguments raise, in some instances, differences of opinion in the evaluation of its application.

The Department asserts that the tribunal should not "second-guess' the judgment of the reviewers and alter the score assigned to Miccosukee in a competitive competition based on isolated comments made in one of eighty applications reviewed.

As with any competitive grant process, objective criteria are evaluated by a panel of reviewers. The role of the tribunal in this process is not to reevaluate the application and supplant the judgment of the reviewers with its judgment. The role of this tribunal is to ensure that the process of selecting successful grant recipients is fundamentally sound.

Regarding the scoring aspect, it was held in Sisseton Wahpeton Community College at 6, that the score rendered by a reviewer with respect to each criterion should not be disturbed unless significant error exists. Such a standard recognizes that reasonable persons may differ in their evaluations and yet permits correction in the event of significant error.

In the instant case, the overall ratings by the three panel members ranged between 20 to 26 points out of a possible 105 points which reflect a general consistency in their overall evaluations.See footnote 5 5 After standardizing, Miccosukee had an average score of 44.48 which reflected individual standardized scores of 41.76, 42.94, and 48.75. This score produced a ranking of 67th of the 80 applications reviewed. I find that the scores of the reviews were reasonable and justified. Inasmuch as no significant error exists, no modification in Miccosukee's score is warranted.

Miccosukee argues that, in scoring the "need" criterion, the reviewers provided inconsistent numerical scores. As such, Miccosukee asserts that an adjustment should be made to reflect a consistent evaluation of its application. The Department counters that the overall scores of the reviewers were consistently low and that Miccosukee argument is based primarily on a difference of opinion with the review panel. As such, the Department argues that no modification of the score is warranted.

While Miccosukee argues that the numerical scores assigned by the review panel were inconsistent with each other, it does not provide a basis for its argument. In this regard, Miccosukee fails to identify specific instances to warrant a modification in a particular score. Nonetheless, under the "need" criterion, the reviewers evaluated the applications based on a maximum score of 15 points.See footnote 6 6 The reviewers, in determining the degree to which the application satisfied this criterion, rated the application as 2, 3, and 7. As such, two of the reviewers evaluated the application in the low to mid-range of inadequate (1 to 5) while one reviewer rated the application in the low range of adequate (6-10).

In the tribunal's view, reasonable persons may differ in rating an application with respect to specific criteria, even among members of the same panel. As such, differences can be expected. In this case, however, the differences are neither significant nor unreasonable.

Miccosukee also asserts that the reviewers interpreted the criteria in favor of organizations with greater resources. In this regard, Miccosukee argues that the reviewers favored colleges and universities with vast resources to the detriment of smaller tribes with limited resources such as itself. Accordingly, Miccosukee concludes that it could not fairly compete with the larger colleges and universities.
The Department notes that the reviewers selected to participate in evaluating the grant applications possess specific knowledge and experience in providing vocational education to smaller tribes. In addition, the Department indicates that a Native American reviewer was assigned to each review panel. As such, the Department asserts that the reviewers were well-suited to evaluate applications of smaller tribes. In interpreting the criteria, the Department asserts that specific materials and instructions were provided to the reviewers to ensure that the review process was based solely on the applicable criteria.

In the instant case, the reviewers were provided with a uniform set of guidelines to interpret the criteria set forth in the regulations. These guidelines ensure that the process is conducted in a fair and impartial manner to all applicants. To the extent that Miccosukee's argument is viewed as a challenge to the validity of the criteria established by regulation, the tribunal, as noted above, lacks the authority to pass upon such a challenge. See In re Sisseton Wahpeton Communitv College, Dkt. No. 95-86-O, U.S. Dept. of Education (Aug. 7, 1995); In re Smithville R-II School District, Dkt. No. 91-4-I, U.S. Dept. of Education (Sec. Dec. July 27, 1992); In re Lemont Township High School District #210, Dkt. No. 89-48-I, U.S. Dept. of Education (Fete 6, 1992), certified by the Secretary (May 4, 1992).

With respect to the interpretation of the criteria, Miccosukee does not identify the manner in which it believes that the review panel favored colleges and universities. Absent specificity, the tribunal will not speculate as to the basis for this assertion. Accordingly, Miccosukee's argument is rejected.

In summary, the tribunal concludes that there is no basis to void the proposed grant awards or to alter Miccosukee's rating.

II. ORDER

In light of the above, it is HEREBY ORDERED that the appeal by the Miccosukee Corporation is dismissed with prejudice.

Allan C. Lewis

Chief Administrative Law Judge

Issued: March 24, 1997
Washington, D.C.

SERVICE

On March 24, 1997, a certified copy of the attached Decision was sent by certified mail, return receipt requested to the following:

Michael Salem
Miccosukee Corporation
P.O. Box 440021
Tamiami Station
Miami, Florida 33144

Daphna Crotty, Esq.
Office of the General Counsel
United States Department of Education
600 Independence Avenue, S.W.
FOB-10B, Room 5442
Washington, D.C. 20202-2110


Footnote: 1    1 1/ The Department utilized review panels consisting of three reviewers to evaluate the applications. The evaluation was based upon the seven criteria set forth in 34 C.F.R. § 401.21 totaling 100 points and two additional criteria set forth in 34 C.F.R. § 401.20(e) totaling 10 points. See also 34 C.F.R. § 401.20(b). Accordingly, the maximum score was 110 points.

Footnote: 2    2 2/ The numerical scores were standardized and averaged to eliminate the problem of employing more than one review panel to evaluate the applications.

Footnote: 3    3 3/ The review procedures were as follows--

First, review the application in order to establish your overall understanding of the application's purpose and methodology.

Second, systematically read the application in conjunction with each technical review criterion in order to determine how well the application meets the individual review criterion. In most instances, each criterion will have a number of sub-parts.

Third, write your evaluation of the strengths and weaknesses of the application for each criterion.

Fourth, upon completing the written review for each criterion, circle the descriptor on the scale and assign the score that best reflects your rating of the application with respect to that criterion.

Fifth, record your scores on the front sheet, total them, and sign and date the application review form.


Footnote: 4    4 4/ Grants under the Indian Vocational Education Program are governed by 20 U.S.C. § 2313(b)(1)(A)(ii)(I) which indicates that grants under this program are governed by § 102 of the Indian Self-Determination Act, 25 U.S.C. § 450f. Section 102 provides for a "hearing on the record" to resolve disputes such as those at issue herein. 25 U.S.C. § 450f(b). The "hearing on the record" language mandates that the hearing is a proceeding governed by the Administrative Procedure Act. 5 U.S.C. § 554(a).

Footnote: 5    5 5/ The total amount of points which may be awarded is 110. Five additional points are added by Department staff pursuant to 34 C.F.R. § 401.20(e) for applications submitted by tribally controlled community colleges that meet certain established criteria not relevant herein.

Footnote: 6    6
6/ The descriptors associated with the numerical scores under this criterion are--
not addressed ------- 0 points
inadequate ---------- 1-5 points
adequate -------- ----- 6-10 points
good to excellent -- 11 -15 points