A common issue cited by participants during ourresearch was a perceived loss of control of data integrity when transitioning from paper to digital Monitoring, Evaluation, Research, and Learning (MERL).


This post is part of a series on gaps identified during a Design Thinking session held at the MERL Tech Jozi conference held in 2018.


With the removal of traditional “paper-source documents” from the data collection process, MERL Tech users and practitioners want to know they can trust the integrity of data captured and managed electronically. Many MERL professionals are sceptical by nature (it’s part of their job, really), and that’s a good thing –it compels MERL Tech providers to be rigorous and put safeguards in place to reassure users of data integrity.

MERL Tech users and practitioners want to know they can trust the integrity of data captured and managed electronically.

The caution expressed stems primarily from three sources:

1) Legitimacy. How can we be sure that the data captured by fieldworkers are legitimate?

2) Accuracy. Was the information accurately/correctly entered by the fieldworker?

3) Visibility. How can we get visibility on data cleaning activities conducted digitally?

During the workshop, participants noted many of the potential ways that technology can, if correctly utilised, provide not only the same level of comfort that paper provides in terms of data integrity, but go far beyond it.

Concern #1: Legitimacy

Goal: Ensure data collected are legitimate

A skilled data manager can often cast their eye down a paper form and – almost eerily – detect a fieldworker who is fictionalising data. They recognise patterns and telltale signs of fraudulent activity which, for the less experienced among us, are next to impossible to pick up.

This is where MERL Tech can help. Assuming the software tool you’ve decided to use captures some key metadata automatically, you can confirm data validity by looking at a few clues:

Time of entry
What time did data entry occur? It’s unlikely that a form captured outside a fieldworker’s normal working hours is legitimate.

Duration
How long did the form take to complete? If the form duration was much shorter than the average, this could signal a data integrity problem. Some tools allow you to see more detail around the path taken through a form and how long each individual field took to complete.

Location
Where was the form captured? Using GPS, the location where a form was completed can be critical in determining whether it is a legitimate entry. If all submissions come from a suspicious location (such as the local pub or the fieldworker’s home), you have some investigating to do. If it makes sense in your context, some tools allow you to build “geo-fencing” into your forms to ensure that a fieldworker is within a certain geographic area before they can capture data.

If all submissions come from a suspicious location (such as the local pub or the fieldworker’s home), you have some investigating to do.

At Mobenzi, we’ve built visualisations into our tools that allow organisations to quickly check these data quality indicators.

Another good practice is to use multimedia to build additional data integrity checks into your form. For instance, it’s relatively difficult for a fieldworker to fake photos of where they’re supposed to be, or mimic respondents providing verbal consent in a recorded audio clip.

Concern #2: Accuracy

Goal: Ensure data entries are accurate

Building strong validation and skip logic into your form’s design is your greatest weapon in improving data entry accuracy.

A good tool will allow you to build powerful validation and skip logic into your form. A simple example is to implement double-data entry for key fields (e.g. a respondent’s ID number). By creating a validation rule that ID_entry1 must equal ID_entry2, you can cut out most input errors. You can also make use of confirmations and warnings (e.g. if a value is outside of a “normal” range) or ask the same question in two different ways. For instance, for a respondent’s date of birth, create both a date and age field and then use a calculation to ensure they match.

Building strong validation and skip logic into your form’s design is your greatest weapon in improving data entry accuracy.

In the early days of digital data collection, when fieldworkers had relatively little experience with technology, their ability to capture accurate data using “new technology” was a much bigger concern. Nowadays, with near-ubiquitous coverage, most fieldworkers have a decent grasp of mobile technology – sometimes better than their managers. However, an exceptional mobile user interface is still critical.

Most fieldworkers have a decent grasp of mobile technology… (but) an exceptional mobile interface is still critical.

Fieldworkers need to be able to get visual feedback as they interact with the form to ensure what they actually capture is what they’re meant to capture. You should also ensure that the device you choose has a reasonable screen size and quality – budgeting a little more on your devices can yield massive savings when it comes to data cleaning.

Concern #3: Visibility

Goal: Effective management of data integrity post-collection

On paper, the trusty red pen is often the way organisations track changes to the original data captured by a fieldworker. The trouble with this approach is that you can’t always tell who made the change, when it was made, or why.

The trouble with this (the trusty red pen) approach is that you can’t always tell who made the change, when it was made, or why.

A good MERL Tech tool will track any changes that you make to data once it has been captured. At Mobenzi, we automatically store information about who made a change, what the value was changed from/to, and timestamp when the change was made. The user (who must have the necessary permissions to edit data) is also prompted to provide a reason for the change, although this is optional. This information is then displayed when viewing a submission (in red) so that the entire history of data cleaning can be seen.

The bottom line is that the original information captured is never lost or overwritten. Changes are stored separately and “overlaid” one by one. Be wary of tools that don’t provide this type of data auditing capabilities.

Final thought

Until implementing organisations have built sufficient confidence in the legitimacy and high standard of data submitted through electronic channels, it will be difficult for MERL Tech to gain traction – but the evidence supporting digital adoption continues to grow.

If you’re interested in learning more about our findings, the full report can be downloaded from our website.