Inequality at the Kindergarten Starting Gate and Quality at the ECLS-K Starting Gate

“Starting Gate” is an apt metaphor in two senses.  First, this monograph studies the relationship between children’s social background characteristics and their academic achievement at Kindergarten entry.  Second, this is the first book-length publication based on the federal National Center for Education Statistics’ Early Childhood Longitudinal Study, Kindergarten Class of 1998-99.   That’s a mouthful, which more commonly goes by the acronym, ECLS-K.  The valuable nuggets of information the monograph presents barely scratch the surface of the ECLS-K databases.  In an effort to encourage interest in both the book and the databases, I will first review the monograph and then preview the ECLS-K databases.

The monograph includes five chapters.  Chapter 1 explores, via graphs, relationships among race/ethnicity, socioeconomic status quintiles, and the results of a Math test and a Reading test individually-administered to children at kindergarten entry.  The SES variable is a composite of household income, parents’ education level, and parents’ occupational prestige.  The test variables are also statistical composites.  Analyses at the item-source level remain a future matter.  Racially/ethnically, the sample of over 16,000 nationally representative kindergartners were: 61.1% White, 17.6% Black, 14.1% Hispanic, and 2.5% Asian.  The authors chose to combine the 2.4% “Mixed Race,” 1.9% Native Americans, and .5% native Hawaiians into a meaningless category: “Other.” Again, analysis at the source level acknowledging these three groups is a future matter.     

When race/ethnicity is graphed against SES quintiles, Whites trend linearly upward across quintiles from Low=9.3% to High=27%.  Blacks trend downward from Low=33.8% to High=7.5%.  Hispanics likewise trend downward from Low=28% to High=9.8%.  Asians form a more complex distribution: Low=13.5%, Low-mid = 11.5%, Middle=14.6%, High-mid=20.8%, and High=a whopping 39.5%.

When race/ethnicity and SES quintiles are graphed against Math and Reading Scores, the results are:

Here, the matter of screening out non-English speaking children in the achievement testing warrants note.  Only 33% of Hispanic and 48% of Asian children had full test data, and those who were tested were of higher SES than were those who were not tested.  The exclusions were reasonable for ECLS-K purposes.  But the children cannot be “screened out” of kindergartens and are additionally “unequal” at kindergarten entry.

The fact that racial/ethnic differences are larger in Math than in Reading and that Asian children outscore Whites more in Reading than in Math hint that something more than race/ethnicity and/or SES is determinative.  The authors peek at a long list of such possibilities in Chapter 2.

Here the findings are presented as descriptive statistics, with some 60 variables again arrayed by race/ethnicity and SES quintiles.  In Chapter 1, the authors concentrated on the Math and Reading test Means, without commenting on the differences in variability (i.e. Standard Deviations) across the race/ethnicity and SES categories.  The descriptive statistics show that in both Math and Reading, Blacks have the most homogeneous scores, with Hispanics, Whites, and Asians linearly more heterogeneous.  I’ve expanded the distributions (assuming a normal curve for expository purposes) to illustrate two pedagogically significant points:

Math

Reading

First, the distributions overlap considerably.  Second, the differences in variability “push the categories together” at the low end of the distributions and “pull them apart” at the upper end.  Rather than differential “disadvantage,” what we see is differential “advantage.”  Math and Reading skills are learned.  The opportunities for the instruction that transpired for the preschoolers to acquire the skills differ predictably by race/ethnicity.  (Parallel relationships hold for SES.)  However, since conventional kindergarten instruction does not presume that entering pupils have learned any Math or Reading, the teaching challenge for those who have learned little is much the same across all race/ethnicity categories.

For the descriptive categories other than test scores the relationships are strikingly linear across SES quintiles and are paralleled by White-Asian vs. Black-Hispanic distinctions:

Linearly increasing percentages from Low to High SES quintiles: Suburban residence, Number of books, records/tapes/CD’s, Own home computer, Parent and child play games, build things, sing songs, make art, Parent teaches child about nature, tells child stories, reads to child, Child does sports, does chores, reads books outside school, Visits zoo/aquarium, museum, library, Attends plays/concert/show, sporting events, Participates in athletic events, organized clubs, organized performing, Takes dance lessons, music lessons, craft classes, Attends preschool other than Head Start, Parents hold high expectations of higher education for child.

Linearly decreasing percentages from Low to High SES quintiles: Single parent household, Number of siblings, Number of homes lived in since birth, attended Head Start, Number of hours watching TV weekly, watches Sesame Street. (Although the authors gloss over the findings, throughout the report Head Start and Sesame Street have zero-to-negative relationships to the achievement tests.  Hmmm.) 

Although the patterns are highly regular, the magnitude of frequency and the spread across quintiles makes the differences smaller than the pattern regularity might imply.  For example: Takes drama classes ranges from .7% to 3.8%, Parent and child build things ranges from 38.3% to 40.7%, Watching Sesame Street from 72.8% to 51.7%.                 

Chapter 3 presents the summary results of a 7-step hierarchical least squares regression analysis using the Math and Reading scores as dependent variables.  The intent of the analysis is to examine the relationships of race/ethnicity to achievement when statistically controlling for SES and for clusters of variables arrayed as: Child Demographic, Parental Educational Expectation for the child (less than h.s. through MA-Ph.D-M.D), pre-k Care, At-home Activities, and Outside-home Activities.

Mechanically, such analyses are simple.  You plug numbers into a computer using one of several statistical software packages, and the computer spits out the results.  Technically, however, such analyses are fraught with hazards.  The specific results obtained are affected by the order in which the cluster of variables is introduced into the analysis.  Care must be exercised to avoid spurious relationships (e.g. Use of multiple SES quintiles rather than a single indicator of SES).  And composite variables (e.g. factor analytic scores) are dubiously interpretable. 

That said, taken with a statistical grain of salt, the results comparing the first step (race/ ethnicity only) with the seventh step (Adding 38 additional variables as statistical controls) are likely robust, and can be summarized as follows

                                    R² (=  % of Variability Accounted For)

                                     Race                 Race+ 38 Variables

          Math                   .079                              .307

          Reading              .045                              .277

Introducing variables other than race/ethnicity into the equation does enhance the predictability of the Math and Reading test scores, and it does statistically “explain away” a good part of the race/ethnicity relationships.  The authors make more of the results of the intervening steps than I feel comfortable in doing.

Chapter 4 presents several multivariate analyses exploring the relationship between race/ethnicity, SES, and various measures of “quality” of the schools which the entering kindergartners attend.  Although again the authors’ structuring of composite variables defining “school quality” will raise the eyebrows of statisticians and others, the summary finding appears robust:

Whether defined by less favorable social contexts, larger kindergarten classes, less outreach to smooth the transition to first grade, less well-prepared and experienced teachers, less positive attitudes among teachers, or poor neighborhood and school conditions, children from less advantaged social background begin elementary school in lower-quality institutions (p. 76, Italics in original).

That finding, together with the earlier described finding that entering kindergartners’ achievement in Math and Reading differs predictably by race/ethnicity, but can in large part be accounted for by SES and other environmental variables, constitutes the monograph’s “bottom line.”  Whether or not that’s a “big deal” depends upon one’s point of view.  That this kind of inequality exists in schooling has been recognized for several decades.  Documenting that it exists at the “starting gate” is “news.”  Identifying differences in status variables such as race and SES begs the instructional question of how to leave no child behind in school.  More exploitation of the ECLS-K databases is needed.

Chapter 5, “Conclusions and policy recommendations” briefly reviews the findings, highlighting the points that impressed the authors.  Because these points are reflected in the foregoing, I’ll not repeat them here, but rather move on to the ECLS-K story.

If you are:

• A graduate student looking for a thesis or dissertation
• A professor looking for a journal publication or conference presentation
• A college instructor looking for a constructive basis for teaching statistics/research/analysis in education
• Or if you are just curious about young children, their families, their pre-school experiences, their teachers, and their schools

ECLS-K is for you.

The set of databases that constitute ECLS-K are monumental and magnificent.  The endeavor began to study some 20,000 nationally-representative children entering kindergarten in the fall of 1998.  Children were individually tested with measures of “Direct Cognitive Achievement” at the beginning and end of both kindergarten and first grade.  The study also obtained interview data from each child’s “parent” and questionnaire information from teachers and principals.  These data have been organized into ready-to-analyze form using any one of several standard statistics software packages.  All of this is currently available, free, on three CDs from the federal National Center for Education Statistics at http://nces.ed.gov/pubsearch/getpubcats.asp?sid=024. 

The study collected another wave of data in spring 2002 when the cohort was finishing third grade.  These data are scheduled to become available this fall.  Data collection for the cohort will conclude in spring 2004 when the children will be finishing fifth grade.

A companion “Birth Cohort” (ECLS-B) endeavor was initiated with 13,500 infants born in 2001.  Data for the children were collected in 2001-02 when they were age 9 months.  Another wave of data is being collected this year at age 24 months.  Data collection will continue in 2005 at 48 months, 2006-07 at beginning and end of K, and conclude in 2007-08 at beginning and end of Grade 1.  None of these databases have yet been released. Together, the Birth and Kindergarten cohorts constitute an unsurpassed data array regarding young children.

Ironically, what ECLS-K terms “Direct Cognitive Achievement” variables are the least described and most inaccessible sector of the databases.  Although the scope of the testing is reasonable and appropriate, the items per se have not been released, and data on pupils at the item level of detail was not included in the data bases: items were immediately combined into “scores,” losing the item-level information.  Nevertheless, since the scores are presented in both “item response theory” and “proficiency” form, a sense of achievement in Math, Literacy, and General Information can be obtained one-step-removed from the item level.  Furthermore, supplementing the “direct” achievement measures are parent and teacher reports of achievement in as much detail as one would wish.

“Parent” interview data are direct, detailed, and extensive.  Included are birth weight and birth prematurity, whether parents were married when child was born, WIC benefits and other health and social services received, parental involvement in various school activities, detailed questions about “family structure.” “race/ethnicity and language,” “home environment, activities and cognitive stimulation,” such details as whether the family eats meals together, what time the child goes to bed, attendance of religious services, availability of persons providing help in various emergencies, “child care out of school,” “family rules regarding television,” “child health” and “well-being,” “parent education, employment and income,” and “child mobility and plans to move.”

Teachers too went “far beyond the call of duty in contributing to the endeavor.  For example, at the end of the year, Kindergarten teachers completed:

1. A 28-page questionnaire reporting on classroom composition and instructional practices.
2. A 9-page questionnaire regarding personal views on a wide range of topics and personal education background.
3. A 9 page questionnaire for each child in the class, rating the child in 9 areas of “language and literacy,” 5 areas of “general knowledge,” 7 areas of “mathematical thinking,” several areas of “social skills,” and 17 other items comparing the child in various instructionally relevant ways.

ECLS-K Field Assessors visiting each school completed a 3-page questionnaire regarding facility features, security features, and extending to percentage of children observed fighting, laughing, crying, and chatting.

“School administrators” likewise contributed, responding to a 34-page questionnaire ranging from the mundane of school characteristics to the intimate of school climate.

Clearly, the potential analyses are many, but mining the data is not a casual matter.  The First Grade Manual is 377 .pdf pages to be printed from the CD (and read).  The K and Longitudinal Manuals are similarly weighty, but since each Manual has the same structure, they are successively easier to navigate.  The Interview Protocols and Questionnaires are a couple of inches thick when printed.  Altogether, the documentation constitutes a foot-high stack.  But from there, all you need is access to a computer loaded with a standard statistical software package, and you’re in business.  All you have to figure out is what you want to analyze and how to interpret the statistics the computer provides.  ECLS-K has done all of the messy drudge work for you and provided you a wide latitude of relevant variables to play with.

Given the current popularity of “evidence-based education” and the focus of federal legislation on the ECLS-K age span, one would think that the education profession would be all over the databases.  That hasn’t happened.  The few analyses that have been performed barely scratch the surface.  The evidence that the initial reports provide is highly informative, but there is no evidence that any of the information has affected policy or practice in any way at federal, state, or local levels—(an interesting item of evidence in its own right).

The bright aspect of the situation is that the data are there to be exploited and that further data are forthcoming.  Let the analyses begin!

The foregoing synopsis does not do justice to either the richness of the ECLS-K data or the competent manner in which NCES and its contractors planned and are executing the data collecting, processing, and documenting.  Even with the less-than-optimal “direct achievement” variables, the databases represent a huge advance in answering the question, “What is it our children are learning?”  Even more important, they provide the wherewithal to determine what home and school practices determine that learning. 

It behooves federal Department of Education to ensure that ECLS-K is not a “one-shot study.”  Although it’s right before their eyes, the Department has yet to recognize that ECLS-K is the best evidence base regarding early elementary education ever produced or on the drawing boards.  The next go-around could be implemented even more effectively and at less cost by building on the experience of the initial round.  Doing so is easily the nation’s best bet single effort for accomplishing the intent of No Child Left Behind.  ECLS-K is a prototype for resolving the incongruity in expecting “annual yearly progress” through federal dictates that reverberate in state governments and school districts and then impact on teachers and children with inevitable unintended consequences. Exploiting the prototype will also provide a means of accomplishing the intent more regularly and rapidly than the arbitrary 12 years, which as things now stand will leave some portion of an entire generation of children poorly instructed. 

It’s unfortunate that the replication of ECLS-K did not begin earlier.  But fortunately the Kindergarten Class of 2004-5 will still provide a live opportunity to maintain the continuity.