ABOUT THE INDEX
Increasingly, demands are being made on analysts to simultaneously consolidate and correlate data from multiple sources into easily digestible statistical measures that provide an indication on how governments, institutions and even students are performing over time.
“A performance index is a powerful management tool that allows multiple sets of information to be compiled into an overall measure which then provides a quick snapshot of performance.”
As a management tool, the performance index works optimally where the indices are correlated to set institutional targets and where differences between performance and plans exist,factors that may be prohibiting achievement of targets are ascertained and dealt with. The index also allows for benchmarking over time to determine whether desirable gains are being recognized periodically.
Furthermore, the report contains approval ratings of all elected leaders operating within the County namely ; Governor, Senator, Women’s Representative, and Member of the County Assembly. The Member of Parliament’s approval ratings were also included in the overall survey much as they operate under the ambit of national government.
The Index will be the first of its kind and magnitude in Kenya. It provides an excellent baseline from which other evaluation tools can be developed and more importantly provides the county governments and other interested stakeholders with robust statistics by ward on the perception of county residents over county governments’ performance of the devolved functions.
The CountyTrak Performance Index was designed with the core objective of ascertaining county residents’ assessment on the performance of their County Governments against set Key performance indicators and to develop a Citizens Scorecard on the performance of their County governments.
The CountyTrak Index will be conducted periodically to provide a reliable palate for benchmarking. Owing to the extremely vast nature of the data, mining of the information shall be done by demand. Over and above this, an executive summary for general consumption is available.
The CountyTrak Index was conceptualised and funded by Infotrak Research and Consulting.
Kenyans overwhelmingly ushered our current constitution on the promise of Devolution and its associated benefits which they felt would translate to increased grass root access to resources and political power. It’s not therefore surprising that since the election of governors in 2013, majority of people have had extremely high expectations of their county governments.
While most county governments spent the nascent period predominantly laying the ground work and planning aptly in order to deliver on the promises, others grabbed the opportunity not only to lay the operating framework, but also to actualise quick wins that resonated with the public.
Two years into devolution and twenty seven months to the next elections the time for concerted action is here. County governments need to fully decipher the perceptions of their residents and develop strategies that are concomitant with their expectations. It’s against this back drop that Infotrak research & Consulting developed The CountyTrak Index.
At the County stage, each County was allocated a quota sample of n=600 and which translated to anto a margin of error of +/- 4 at 95% degree of confidence. The overall sample was n=28,200.
The CountyTrak Index was conducted between December 2014 & February 2015
On each quota county sample, the 2009 census was then used to stratify the sample using Population Proportionate to Size (PPS) across the respective wards within the county. The random stratification using a CATI predictive dialer was designed to ensure that each adult Kenyan in the respective wards with access to a mobile phone, had an equal chance of being interviewed. The ward was used as the key administrative boundary.
The predictive dialer settings also ensured fair distribution by age and gender provided the mother database is robust and representative of the universe. Data processing & analysis was done using SPSS
Infotrak adopted a participatory approach in designing the research instruments and training manuals. Between March and August 2014, various questionnaires were adopted and county reports prepared for select counties. These reports were shared with the governors of those counties and their critique and objective comments internalised to design the optimal tool that encompassed what would be deemed by them as a fair assessment of their performance. The refined tool and reporting was also shared with development partners and donors working on devolution. Their comments were also internalised and adopted. Finally, the tool was subjected to rigorous refining and crystallizing of indicators that would fall under each devolved function. In the end the CountyTrak Index tool consisted of thirty three indicators and five approval rating questions. The tool also asked residents to indicate the key issues of concern for the county amongst other things.
The final questionnaire was structured with likert type ranking questions applied on each key performance indicators; ie on a scale of 1-10 where 1 is very poor and 10 is excellent.
The final draft CountyTrak questionnaire was piloted between October and November 2014. After the pilot, several fundamental changes were made to ensure the survey was expedited with efficiency. To begin with, the Swahili version of the questionnaire was simplified for easier consumption. Secondly, the indicators were reduced from fifty to thirty three. Last but not least the questionnaire length was shortened to ensure that respondents do not suffer irritation or fatigue during the questioning.
Data Collection team & Training
The data collection team/CATI interviewers were selected from a pool of nearly two hundred individuals who had prior experience in CATI research. The team was trained using both classroom method and hands on approach to gauge their suitability for the survey. During the pilot, their dummy interviews were listened to by professional researchers and best practice recorded and used for illustration whilst weaknesses were rectified but also recorded and used for correction. In the end, the CountyTrak CATI team consisted of individuals who were well versed with the tool, well prepared to handle all types of telephone conversations and well-motivated to handle any challenges that emanated from the calls.
During the training, the team was also provided with a training manual that consisted of frequently asked questions (FAQs) and any other critical issues about the survey that they needed to remember.
The CATI team had on call supervisors and IT support to assist them with any challenges that they faced.
Data was collected through Computer Assisted Telephone Interviews and data processing & analysis was done using SPSS.
Kenya with a current subscriber base of 33million and mobile penetration of 82.6% (according to CAK) provides an excellent opportunity for the adoption of technology in data collection.
Each Infotrak CATI interviewer (ICI) sits behind a CATI station and initiates the interview sessions.
The ICATI uses a predictive dialer which randomly selects and assigns numbers to each ICI.
Limitations of the Study
Like every evaluation tool, the CountyTrak Index isn’t without shortcomings and limitations. To begin with, comparing counties is an obviously very challenging effort given that county governments are operating with advantages and disadvantages that may be decades old; including the presence of natural resources and political marginalisation.
Secondly, Opinion leaders from the counties in the northern frontier which have hitherto been perceived as marginalised feel that their populace is less exposed and therefore less informed. Whilst we accept this as a limitation, we are quick to point out that illiteracy is not concomitant with unintelligence. That a person may be unable to read does not mean that they can’t opine on the issues affecting their lives in the county.
Thirdly, the introduction of technology in data collection has naturally been met with skeptism both from some respondents who find the telephone interviews strange and the general public who are yet to fully synthesize how the technology works. Further still, there are those who erroneously believe that mobile phone ownership and subscription is the preserve of the rich.
Be that as it may, Kenya with a current subscriber base of 33million and mobile penetration of 82.6% (according to CAK) provides an excellent opportunity for the adoption of technology in data collection. CATI still has other limitations which include the fact that the questionnaire cannot be long as respondents get impatient with interviews longer than 15 minutes. The call rates are still extremely expensive making the method only slightly cheaper than face to face. The advantages of CATI are accrued more in accuracy and quality control because it is easier to establish with certainly when, how and with what quality the interview was conducted.
|County Planning & Development|
|County Infrastructure ; Roads & Transport|
|Control of Pollution|
|Conservation of Environment and Natural resources Functions|
|Cultural activities, Public Entertainment and Amenities|
|Education: ECD and Polytechnic education|
|Trade Development & Regulation|
|Animal Control and welfare|
|Making it difficult for people to access drugs and pornography|
|Management of public funds|
|Empowerment of youth, women & persons with disability|
Condition of life of county residents
Happiness index of the county residents
Optimism index of the county residents