Increasingly, demands are being made on analysts to simultaneously
consolidate and correlate data from multiple sources into easily digestible
statistical measures that provide an indication on how governments,
institutions and even students are performing over time.

“A performance index is a powerful management tool that can allow multiple
sets of information to be compiled into an overall measure which then
provides a quick snapshot of performance. It could also be based on one key
parameter on overall performance”

As a management tool, the performance index works optimally where the
indices are correlated to set institutional targets and where differences
between performance and plans exist, factors that may be prohibiting
achievement of targets are ascertained and dealt with. The index also allows
for benchmarking over time to determine whether desirable gains are being
recognized periodically.

Furthermore, the report contains approval ratings of all elected leaders
operating within the County namely; Governor, Senator, Women’s
Representative, and Member of the County Assembly. The Member of
Parliament’s approval ratings were also included in the overall survey much as
they operate under the ambit of national government.

Introduction

Kenyans overwhelmingly ushered in our current constitution on the promise of devolution which they felt result in increased grass root access to resources
and political power. Indeed people’s expectations of county governments continue to be extremely high.

While most county governments tend appreciate this fact and try to capitalise on low hanging fruits as soon as they kick off, others prefer to spend the nascent stages laying the ground work and planning elaborately for delivery. Irrespective of leadership style, what remains critical is that people feel the
impact of devolution in their daily lives.

Questionnaire Design

For the inaugural Countytrak Index, Infotrak adopted a participatory approach in designing the research instruments and training manuals. Between March and August 2014, various questionnaires were adopted and county reports prepared for select counties. These reports were shared with the governors of those counties and their critique and objective comments internalised to design the optimal tool that encompassed what would be deemed by them as a fair assessment of their performance. The refined tool and reporting was also shared with development partners and donors working on devolution. Their comments were also internalised and adopted. Finally, the tool was subjected to rigorous refining and crystallizing of indicators that would fall under each devolved function. In the end the CountyTrak Index tool consisted of thirty three indicators and five approval rating questions. The tool also asked residents to indicate the key issues of concern for the county amongst other things.

The final questionnaire was structured with likert type ranking questions applied on each key performance indicators; ie on a scale of 1-10 where 1 is very poor and 10 is excellent.

In July 2019, the tool was further refined to rename the functions to mirror the adaptation that most counties currently use.

The data collection team/CATI interviewers were selected from a pool of nearly two hundred individuals who had prior experience in CATI research. The team was trained using both classroom method and hands on approach to gauge their suitability for the survey. During the pilot, their dummy interviews were listened to by professional researchers and best practice recorded and used for illustration whilst weaknesses were rectified but also recorded and used for correction. In the end, the CountyTrak CATI team consisted of individuals who were well versed with the tool, well prepared to handle all types of telephone conversations and well-motivated to handle any challenges that emanated from the calls.

During the training, the team was also provided with a training manual that consisted of frequently asked questions (FAQs) and any other critical issues about the survey that they needed to remember.

The CATI team had on call supervisors and IT support to assist them with any challenges that they faced.

Like every evaluation tool, the CountyTrak Index isn’t without limitations.

To begin with, comparing counties is a challenging effort because county governments are operating from different starting points.

Secondly, the perceived notion that respondents from marginalised and rural communities have lower comprehension of issues is often argued to imply that because they are less exposed they are also less informed. Whilst this argument is valid, it’s imperative to note that illiteracy doesn’t equate unintelligence. Just because someone can’t to read doesn’t mean they can’t opine on issues that affect their lives within the county.

Thirdly, the introduction of technology in data collection has naturally been met with scepticism both from some respondents and  stakeholders alike who are yet to fully synthesize how the CATI technology works. Further still, there are those who erroneously believe that mobile phone  ownership is the preserve of the rich.