Impact

Our users make a difference

Sign up
Download

\o/ stats

10000

Downloads

Developing a Monitoring and Evaluation Framework for FrontlineSMS

By Juliana Bedoya Carmona, Monitoring and Evaluation intern

I have recently joined the FrontlineSMS team in Nairobi as part of an ongoing Monitoring and Evaluation Project rolled out by FrontlineSMS in collaboration with Tufts University Team; Bronwyn Cook, Shuvam Dutta, Amanda Meng and Julie Younes, who are also part of FrontlineSMS Heroes. So far, my experience in Nairobi has been very enriching. Getting to know the FrontlineSMS Nairobi team, meeting FrontlineSMS users and finding out about other M4D and ICT4D start-ups working out of Nairobi’s iHub, has been a very exciting and valuable learning opportunity.

Since December 2011, FrontlineSMS, in collaboration with a multidisciplinary group of graduate students at Tufts University (from the Fletcher School of Law and Diplomacy and the Department of Urban and Environmental Policy Planning) and under the direction of Dr. Jenny C. Aker, has engaged in the development of a new Monitoring and Evaluation framework. As part of this process, I have recently joined the FrontlineSMS team in Nairobi to roll out a test phase of the M&E framework that includes working closely with user organizations in the field, receiving their feedback and analyzing preliminary data provided by them to further improve the framework as it currently stands. The objective of the M&E framework is to capture data that illustrates what type of impact FrontlineSMS may have in user organizations’ everyday activities in terms of cost, time and communications efficiencies savings. By developing and testing this framework we aim to better understand and improve the ways in which FrontlineSMS may effectively advance towards achieving its mission; that of lowering barriers to transformative social change by using mobile technologies.

The FrontlineSMS:Credit team and Juliana visit Safe Water AIDS Project in Kisumu while Collince showcases their social marketing interventions. Photo Credits: Rick Roxburgh

NGOs and other non-profit organizations have used FrontlineSMS in a wide variety of contexts and with myriad goals in mind. From real time malaria diagnosis in Cambodia to tackling violence against children in Benin, FrontlineSMS is being used in over 80 countries to lower communication barriers and facilitate social change around the world.  But how exactly does FrontlineSMS transform everyday operations of user organizations and what are the dimensions of these changes?

Measuring the impact of information and communication technologies for development (ICT4D) is drawing increasing attention as more organizations engage with technologies to help them press forward their organizational goals. There are, however, specific challenges linked to monitoring and evaluating efforts in this particular area. Often organizations implementing mobile technology tools such as FrontlineSMS in their operations do not set out from the start to record changes related to the incorporation of this technology. From the viewpoint of organizations providing the technologies, such as FrontlineSMS, tracking performance of their product often proves difficult as they do not have direct access to the data showing efficiencies or changes brought about by the implementation of a given technology. Given that FrontlineSMS is open source software it is also not easy to keep track of the innovative ways in which user organizations are implementing the software in their everyday operations, either as standalone or in combination with other applications.  Furthermore, gathering enough data in order to draw confident enough conclusions depends on the number of organizations willing to participate in the M&E framework, fill out surveys and provide feedback that helps tailor the evaluation instruments and selected indicators. As a result, the process of developing an M&E framework for FrontlineSMS has been a gradual and participatory one involving FrontlineSMS staff members, researchers and finally user organizations.

FrontlineSMS has been keen to listen to user organization’s experiences of using the software through its annual user survey, the online community space and the development of case studies about specific organizations. The new M&E framework builds on these tools and incorporates additional ones to help draw a more complete picture of the changes that organizations notice after implementing FrontlineSMS, and how these compare among different user organizations. The elements of this new M&E framework include a logical framework, a new survey and the development of new case studies that provide additional qualitative evidence to help refine the tools that have been produced so far.

FrontlineSMS:Credit at ARO FABlab in Kisumu. Photo credits: Rick Roxburgh

In the next couple of days we are looking to finalize a list of user organizations willing to partner with FrontlineSMS in this effort. By including user organizations in the development and testing of the first phase of the M&E framework we aim to improve the design of the system in order to capture as much rich data as possible, which will then be useful to the FrontlineSMS team and our users. We anticipate that the M&E framework as well as its results will contribute to improved understanding of the usability, benefits and challenges of implementing FrontlineSMS software; providing a clear picture of the types of costs, time and communication efficiencies that could be achieved by using FrontlineSMS; and offer a basic framework for user organizations to track their own communications, cost and time management performance in those areas in which they implement FrontlineSMS.

If you would like to find out more about our M&E framework please contact us at info@frontlinesms.com . We are particularly keen to hear from new and existing users of our software eager to measure the impact FrontlineSMS is making to their work.

Tags: , , , , , ,

Trackbacks/Pingbacks

  1. Developing a Monitoring and Evaluation Framework for FrontlineSMS | Global Health Hub: news and blogosphere aggregator - 24 July 2012

    [...] The rest is here: Developing a Monitoring and Evaluation Framework for FrontlineSMS [...]

  2. Josana Academy Becomes the First School to Use PaymentView « frontlinesms - 23 August 2012

    [...] While at Josana Academy, we also sat down with the head teacher to ask questions about how things work without PaymentView and easy-to-use SMS announcements. By asking how parents currently pay their children’s school fees, what percentage of fees remain unpaid at the end of the term, how many students drop out of school due to financial hardship, how much it costs to issue paper notices, how often the school has parent meetings, and the current attendance rate, we got a good picture of the status quo. We will repeat these questions in three months, which will help us to understand how using PaymentView has made a difference. FrontlineSMS is implementing this monitoring approach across its work, including by providing tools to users to help them to undertake similar studies on their own or with our help. You can read more about this in a blog post from our Monitoring and Evaluation Intern, Juliana. [...]

Leave a Reply

Connect with us

Stay in touch

 

downloads