Why does ASER test children at home and not in school?
The ASER survey generates estimates of schooling and basic learning status for ALL children in rural Pakistan in the age group 5-16 through household based assessment. Household based assessment is the only way to ensure that ALL children are included. This includes children enrolled in different types of schools (government, private, and other kinds) as well as children not currently enrolled in school. Moreover, in our geographies where the non state sector(private) is so wide spectrumed from non-formal (established formally) to madrassas to non-elite non -registered to registered affordable to elite private schools, the household becomes a great catchment space to cover it all. Also school based assessments of student learning will leave out non-attending children who may have poorer learning levels. ASER Pakistan not only tests every child falling in the age group of 5-16 but also surveys the local government and local private schools for facilities, teacher qualification and presence, students attendance, multi-grade or single grade learning, grants to schools etc to provide with a broader picture.
What is the sample size of ASER? How does this compare with other large-scale surveys?
ASER aims to generate district level estimates of children’s schooling status, basic reading and arithmetic. Each year, ASER reaches to entire Pakistan including 138-144 rural districts across Pakistan while also including some of the urban areas into the sample. In each district, 30 villages are selected and in each sampled village, 20 households are randomly selected. This gives a total of 30 x 20 = 600 households in each rural district. Depending on the exact number of districts surveyed, a total of almost 85,000+ households across the country are sampled for each year’s ASER. In every surveyed household, all children in the age group 3-16 are surveyed and children age 5-16 are tested in basic reading and arithmetic. A total of approximately 250,000+ children are surveyed each year.
Why does ASER aim to generate district level estimates?
Mostly official statistics in Pakistan produce estimates at the national and provincial level. However, planning and allocation of resources/budget is done at the district level. It is therefore important to come up with estimates that are representative at district level. ASER serves this purpose by providing gender disaggregated data set each year for robust analysis and planning.
How are the villages selected?
In each district, villages are randomly selected using the village directory of the 1998 Census. Since villages vary by population, sampling is done using the PPS (Probability Proportional to Size) sampling technique. In each year’s ASER, the 30 villages surveyed in a district comprise 20 villages from the previous year’s survey and 10 new villages are added. The 20 old villages and 10 new villages give us what is known as a “rotating panel” of villages, which generates more precise estimates of change. Having a rotating panel of villages means that every year some old and some new villages are included, which ensures that there is both continuity and change in the sample from previous years.
How can I find out which villages have been surveyed?
The data is public and available on the website for anyone to use
Do the ASER estimates for a district also apply to individual villages in that district?
No, they don’t. ASER estimates for a district are representative at the district level, and provide a snapshot of children’s schooling and learning status for the district as a whole. The sampling is not representative at the village level, and the situation in individual villages can be different.
Who designed this sampling strategy?
The ASER Pakistan model is inspired from ASER India. ASER India sampling strategy was designed in consultation with experts at the Indian Statistical Institute, New Delhi. Inputs were also received from experts at the Planning Commission of India and the National Sample Survey Organization (NSSO).
Why is ASER done every year?
ASER is done every year for several reasons. First, in addition to presenting district, state and national level estimates each year, ASER also presents trends over time. Comparable measurements have to be done periodically in order to see how the situation is changing. The ASER measurement is done annually because government plans and allocations for elementary education are made every year. If children’s learning outcomes are to improve, then evidence on how much children are learning needs to be taken into account during the process of review and planning each year. Second, longer gaps between assessments can have serious implications for children currently in school. It is well known that falling behind in school often leads to dropping out altogether. If several years go by between assessments, opportunities are lost to take rapid corrective action in order to ensure that children who are falling behind are able to catch up. Third, it takes time to shift the focus from schooling to learning. When ASER began in 2008, the issue of children’s learning was rarely discussed. But after five years of ASER, the topic of children’s learning and quality of education is very much on the national agenda.
What are the guidelines that are followed in developing the reading and arithmetic assessment tools?
By design ASER is a ‘floor’ test which aims to evaluate children’s early reading and basic arithmetic ability. The content of the reading assessment as well as numeracy is based on grade 2 and grade 3 level curriculum.
Are the reading assessments comparable across different languages?
The ASER reading tool is developed for Sindhi and Pashto too along with Urdu.
Why are all children in the age group 5 to 16 assessed with the same tools? Why does ASER not assess children at their grade level?
All children are assessed with the same tools as the objective of the ASER survey is to ascertain whether or not children have attained early foundational skills in reading and arithmetic. This is irrespective of age or grade level. It is not designed to be a grade-appropriate assessment but rather it is designed to provide an understanding of school-aged children’s early reading and basic arithmetic ability.
When (Month of year) did you conduct the assessment? What was strategic about this timing?
ASER Pakistan is conducted each year during September-November and the final report is launched in mid January. The survey starts taking place in September as it’s a good time when schools are reopened after summer vacations. The chosen time for report launch is mid January as the planning for education policies and the development budget for the next fiscal year by the government takes place after January. By that time, ASER findings are disseminated at national, provincial, district and village level.
Why does ASER use volunteers? Are the volunteers capable and well trained to do the survey?
ASER is a citizens’ initiative, implemented by partner organizations in every rural district across the country. One of the major aims of the survey is to generate awareness and mobilize people around the issue of children’s learning. The entire design of ASER thus revolves around the fact that it aims to reach and involve ‘ordinary people’ rather than experts. All tools and procedures are designed to be simple to understand, quick to do, and easy to communicate. Procedures for ensuring the quality of data have evolved over several years. Typically ASER volunteers are given 3 days of training. One of these days is spent practicing all ASER steps and procedures in the field. At the end of the training, a quiz is conducted to ensure that volunteers have understood the key elements of ASER. In addition, volunteers are monitored when the survey is in the field; and a substantial proportion of villages are rechecked by ASER teams. In ASER 2013, for example, more than half of all surveyed villages were either monitored or rechecked or both.
Were the data collection/measurement tools pretested before the actual data?
Yes, ASER goes through an intensive tool review and testing process every year. Assessment experts from all provinces are called in for a review meeting where they review ASER Tools in the light of curriculum and biasness. After incorporating suggestions from experts, the tools are pilot tested. Tools are further shared by the assessment experts, after pilot testing with results.
Who supervised the data collection process, and what supervision methods were used?
ASER Survey is monitored with the help of partners (other than the survey implementation partners) and ASER Teams. 80% of the surveyed districts are monitored on the day of training as well on the day of survey to ensure the presence of master trainers and volunteers. Call centre is also set up in head office where volunteers and masters trainers are called on daily basis.