Global efforts have succeeded in identifying people living with HIV (PLHIV) and linking them to antiretroviral therapy (ART),1 however, improving the quality of care received remains a challenge. PLHIV require ongoing care, which includes monitoring their response to ART. The World Health Organization (WHO) recommends viral load (VL) testing to check whether the treatment is working,2 particularly in resource-limited settings where access to genotypic drug resistance testing is still limited. Monitoring VL in PLHIV on ART can help to develop quality personalized plans for HIV treatment and care which is important for achieving good health outcomes within this population.2
Unfortunately, although many governments have adopted the WHO guidelines, VL testing coverage remains suboptimal; particularly in sub-Saharan Africa,3–5 a region where nearly 70% (26 million) of PLHIV live.1 In 2017, less than 75% of those on ART had received a VL test in the region.4 In Malawi—a country in sub-Saharan Africa with a per capita GDP of $338 and classified as the second poorest by the World Bank in 2017—over 0.8 million people are on ART, 6,7 but in 2017 only 60% received a VL test.8 At the time of the study, the Malawi Ministry of Health recommended VL testing at 6 months, 24 months, and then every two years after ART initiation,9 but identifying and successfully testing clients who reached these milestones was challenging. Some of the barriers to VL testing in Malawi and other countries in the region include human resources shortages to do the VL tests, lack of demand for VL tests, poor laboratory infrastructure, and suboptimal client flows.10–13
To overcome some of the barriers and improve VL testing coverage, the Malawi government is working with several development partners, including the Baylor College of Medicine Children’s Foundation Malawi (henceforth, Baylor-Malawi). Baylor-Malawi, through the USAID/PEPFAR-supported program Tingathe (meaning “together we can”) has been helping the Ministry of Health to strengthen HIV treatment, care, and support including VL testing in seven districts for more than a decade.14–16 To address gaps in VL testing coverage, Baylor-Malawi’s initial remediation steps in early 2018 included introducing a focal person for VL testing services, training healthcare workers on VL testing, and encouraging PLHIV to request VL tests, but coverage remained suboptimal. Additional efforts in October 2018 included a program directive instructing all sites to allocate a Community Health Worker (CHW) to screen all clients’ health passport books to identify clients due for a VL test, change client flow at the ART clinics to allow for earlier collection of VL samples, and reallocation of existing human resources to increase capacity for VL testing. Unfortunately, an evaluation in December 2018 revealed that significant gaps in the VL testing process remained and VL testing coverage was still suboptimal. We hypothesized that the sites did not fully implement the recommended changes due to insufficient supervision. This hypothesis would be consistent with previous quality improvement efforts which included supervision of the viral load testing process in three sub-Saharan African countries—Swaziland,17 Ivory Coast,18 and South Africa19–21—but did not quantify the effects of the supervision alone.
Several studies have reported that supervision in healthcare improves the process and quality of care,22,23 but in many African settings, the supervision is often not adequately performed. In these settings, key barriers to supervision include cost, lack of skilled supervisors, logistic challenges of transporting supervisors to health facilities regularly,24–26 geography,27 and time.28 To overcome these barriers and improve supervisory capacity in the VL testing process, Baylor-Malawi decided to leverage existing easy-to-use technology which hitherto has not been used extensively in African settings, particularly where logistical and geographic accessibility challenges are significant barriers to in-person supervision.
Baylor-Malawi initiated remote supervision of the VL testing process via WhatsApp in Mangochi—a rural district in Malawi where logistical and geographical barriers to supervision are some of the most challenging in the country.29 WhatsApp is a user-friendly and free mobile instant messaging application that only requires internet connectivity and can be used on any phone with the ability to connect to the internet. The application allows users to send text messages, pictures, videos, as well as voice recordings.30 This platform allowed for real-time reporting of data and challenges at the clinics, as well as timely feedback from supervisors to address those challenges. We expected the WhatsApp platform to provide an opportunity for more regular and consistent supervision without imposing additional costs to Baylor-Malawi. The WhatsApp platform is already widely used and continues to grow in Malawi,31 so the additional cost and learning curve to apply this platform for supervision was deemed to be minimal.
In this study, we evaluate the impact of remote supervision via WhatsApp on VL testing coverage using a quasi-experimental study design. Since facility characteristics can influence an intervention’s implementation,32 we also evaluate whether the effects of the remote WhatsApp supervision vary by facility type (primary vs. secondary health facilities) and ownership (by private and faith-based organizations vs. government) . These facility characteristics may define the availability of the services, number of healthcare workers, health facility infrastructure, as well as the quality of care and health facility leadership.
The present study will provide evidence on whether or not remote supervision via WhatsApp can be used to overcome logistical and geographical challenges to supervision and optimize viral load monitoring. This study was conducted before the COVID-19 pandemic. To prevent further spread of COVID-19, both physical and social distancing are encouraged; this has limited health services delivery to only the most essential and therefore has greatly curtailed important but potentially high-risk activities such as in-person supervision. Therefore, understanding whether remote supervision utilizing an already commonly used and free platform such as WhatsApp can help improve HIV service delivery is even more important now.
The intervention was implemented in Mangochi, a district in Southeastern Malawi with a large catchment population and a disproportionately high HIV/AIDS burden. The district is part of the great African rift valley at the southern end of Lake Malawi with a total land area of 6,273 km2 that is punctuated by highlands and hills. Thus the district’s topography comprises the coastal plains and the hilly-forested areas rising above the plains making most parts of the district hard-to-reach29; therefore, regular in-person supervision is very challenging. On average, it takes two hours to complete one-way travel from Baylor-Malawi offices in Mangochi to nearly half (15/37) of the facilities in the district. Unfortunately, these remote health centers are not all in the same direction; they are scattered all over the district. Further, Mangochi has a high adult HIV prevalence (>10% compared to a national average of <9%),33 and low VL coverage (60%) in 2018. The district has 61,200 PLHIV on ART, representing 8% of Malawi’s overall ART cohort. Most facilities are managed by the Ministry of Health, but we also included facilities managed by the Christian Health Association of Malawi (CHAM) and private owners. All sites provided free ART and VL testing services in line with the Malawi Ministry of Health guidelines.
Routine VL testing services, both pre- and post-intervention
VL samples are collected using dried blood spot (DBS) on filter paper by HIV Diagnostic Assistants (HDAs)—a cadre of lay health workers specially trained to provide HIV testing services. At the time of the study, the Ministry of Health recommended VL testing at six months, 24 months, and every 2 years after ART initiation. Samples are forwarded to the district laboratory via regular motorcycle transport and from there to the central laboratory for analysis.
To enhance the supervision of VL testing services, we created WhatsApp groups at each site to facilitate data reporting and communication. The groups comprised Tingathe staff at three levels: the site (site supervisors) as supervisees, the district (HIV Program Officer, Clinical Mentors, and the Medical Coordinator), and the Tingathe head office (Medical Manager and Senior Medical Officer) as supervisors. The supervisees were asked to post the following information on the WhatsApp group daily at 10 am and 4 pm: 1) the number of clients at the ART clinic, 2) the number of VL samples collected, and 3) any challenges at the clinics. Supervisors were responsible for reviewing this information, noting gaps, and providing feedback/guidance to enable the sites to make changes on the ground in real-time. Only aggregated data were shared on the WhatsApp platform.
We defined supervision as the provision of monitoring, guidance, and feedback by senior or more experienced staff to junior or less experienced ones on VL testing matters.22,34 We hypothesized that the supervision, which was exclusively provided remotely, would improve VL testing coverage because of increased interaction between staff at the clinics (supervisees) and their supervisors and a more accurate representation of situations at the clinics since data reporting and feedback was in real-time.35,36
We implemented the intervention in three phases, starting with sites with the lowest VL coverage and high client volumes. In phase one (December 2018 to January 2019), we rolled out the intervention in six sites. In phase two (February 2019 to March 2019), we rolled out the intervention in 19 sites. The third phase was in April 2019 in 11 sites.
VL testing data, routinely collected by the program, came from the Ministry of Health’s VL testing registers while other Ministry of Health records provided data on classification of facilities according to service level and facility ownership. The VL testing registers, available at each site, provided the number of routine VL samples collected, aggregated by site and month. We checked the monthly data against the data reported via the WhatsApp groups. Only routine VL tests were included; follow-up and targeted VL tests were excluded. The Ministry also provided data on the target number of VL samples to be collected at each facility, which we used for VL coverage estimation. The Malawi National Health Sciences Research Committee and the Baylor College of Medicine institutional review board granted ethics approval.
VL coverage definition
The dependent variable was weekly VL coverage, defined as the proportion of PLHIV on ART who provided samples for VL testing compared to the number of samples that would be estimated to have been collected in that period. According to the Ministry of Health milestones, at the time of this study, 60% of the ART cohort should be eligible for VL tests per year.9 So, we estimated that, at each site, 5% of the cohort was eligible per month (60%/12) or 1.25% per week (5%/4). However, later in April 2019, the Ministry of Health adopted new VL testing guidelines which changed the proportion eligible for VL testing in a year from 60% to 100%.37 We adjusted the denominator for estimating VL coverage from the date of the new guidelines onward. VL coverage was estimated by site and week and then modeled in regressions as a district average. The weekly VL coverage may exceed 100% because of catch-up VL collection. That is, clients who miss their date of appointment and, therefore a VL sample draw, may come to the clinic later which would increase the number of samples drawn in that week while decreasing the number of samples in their scheduled week.
Study design and statistical analyses
We estimated weekly VL coverage at each site for 35 consecutive weeks. The data were split into two distinct periods: the pre-intervention period (August 2018 to April 2019) and the post-intervention period (December 2018 to July 2019). We emphasize that the overlap between the pre- and post-intervention periods is because the intervention was not implemented at the same time across all the health facilities in the district. Within each of these periods, each site had 17 weeks of pre-intervention and 18 weeks of post-intervention data with varying start and end dates. In the analysis, therefore, all the pre- and post-intervention period data were aggregated separately to derive the pre- and post-intervention data for the whole district.
We evaluated the effect of the WhatsApp supervision using single-group interrupted time series analysis (ITSA) to account for pre-existing trends in VL coverage.38 The design assumes that in the absence of the intervention the trend could have continued unchanged or uninterrupted.39 Therefore, the interruption that occurs after the introduction of the intervention is attributable to the intervention. Although ITSA with a comparison group is recommended to check for historical threats to validity,40 it was not possible to have a control group given that the intervention was implemented at all sites supported by Baylor-Malawi in Mangochi district and therefore might systematically differ from those not supported by Baylor-Malawi. Despite this limitation, single group ITSA still produces valid estimates for causal inference.38 This is because the design involves a pre-post comparison within a single population which limits both selection bias and confounding which often occur because of between-group differences. 39,41 Therefore, ITSA reduces the need to control for typical and slowly changing confounders such as facility characteristics since their effects are fairly constant over time.
We hypothesized that the remote WhatsApp supervision increased both the level and trend in VL coverage and therefore estimated the following model using ordinary least squares regression:
Yt= β0+ β1Tt+ β2Xt+ β3XtTt(1)
Yt is the average VL coverage in week t;
Tt is the time (weeks) since the study began. Its coefficient, β1, represents the rate of VL coverage in the pre-intervention period;
Xt is a binary variable representing the intervention. Thus, Xt=0 during the pre-intervention period and Xt=1 during the post-intervention period. Thus, β2 represents the change in the level of VL coverage immediately after implementing the intervention;
XtTt is an interaction of time and the intervention.42 The coefficient of this term, β3, is the difference in the rate of VL coverage in the pre- and post-intervention periods. Therefore, the overall rate of VL coverage in the post-intervention period is given by β1+β3;
β0 is the level of VL coverage at the beginning of the study.
In sub-analyses, we evaluated whether the effects of the WhatsApp supervision differ on two facility dimensions: facility type (primary or secondary level); facility ownership (by Christian Health Association (CHAM), private, or government).
Model robustness and autocorrelation
We checked the robustness of the model in equation 1 by introducing pseudo-intervention periods before the actual date of the intervention, expecting not to find any interruptions in those periods.43 This tested whether the model produced effects in weeks other than the one in which the intervention was introduced. For meaningful testability, the pseudo-intervention periods were set in the 6th and 12th weeks.
Further, we checked the model for autocorrelation―the correlation between a variable and its previous values―and found that the series were correlated but only up to lag 2 (AIC=9.11, P=0.067). Therefore, all regression analyses incorporated two lagged values of the dependent variable as explanatory variables and Newey-West standard errors which can handle both heteroscedasticity and autocorrelation.44 After running the model (equation 1), we verified (using residuals) that the model accounted for the correct autocorrelation structure using the Cumby-Huizinga test for autocorrelation (χ2=3.17, P=0.075).45 All statistical tests were two-sided and were performed in Stata 14.2.
We performed two sensitivity analyses. First, we limited post-intervention data to the week ending 6 April 2019―the week after which the new VL testing policy was announced―to completely exclude the effects of the new policy. Furthermore, we limited the analysis to facilities that had reported at least seven weeks of post-intervention data at the time the new policy was announced because post-intervention trends were likely to have been established by that time. Second, we excluded from the analyses health facilities with outlying VL coverage, defined as VL coverage lying ±2 standard deviations of the mean district VL coverage in either the pre- or post-intervention period.
We estimated the mean weekly VL coverage for Mangochi district using data from 36 sites that reported weekly VL samples collected during the 35-week study period. Most sites (86%, 31/36) offered only primary care and the majority (61%, 22/36) were government-owned. On average, the district collected 701 VL samples per week and the average VL coverage for the entire series was 98%. The average number of VL samples collected and VL coverage were higher in primary (vs. secondary) health facilities (VL coverage, t=4.44, P<0.001). VL coverage did not differ significantly by facility ownership.
Among the 36 sites that reported data, 34 sites (94%) experienced higher average weekly VL coverage in the post-intervention period; Nkope, a CHAM-owned primary health center experienced a decrease in VL coverage while Malembo, a public primary health center, did not experience any change in VL coverage. Overall, the average weekly VL coverage for the whole district in the post-intervention period was 125% compared with 68% in the pre-intervention period, representing an increase of 57 percentage points (t=13.5, P<0.001) (Table 1). In all, a total of 23,754 VL samples were collected during the study period with nearly two-thirds (15,788) collected in the post-intervention period.
The increase in average weekly VL coverage post-intervention was higher in primary and public health facilities (Table 1). Primary health centers registered an increase of 61 percentage points compared to 36 percentage points among secondary health facilities. Public health facilities registered an increase of 62 percentage points compared with an increase of 47 percentage points among Privately/CHAM-owned facilities.
The main model for VL coverage for the whole district showed that remote WhatsApp supervision was effective (Figure 1). There was an immediate step (level) increase of 58 percentage points (95% CI: 48-68 percentage points, P<0.001) from 58% to 116% following the introduction of the WhatsApp supervision, followed by an increasing and sustained post-intervention trend in VL coverage of 2.5 percentage points per week (95% CI: 1.36, 3.67, P<0.001); before the intervention, VL coverage was declining. Materially, of the 15,788 VL samples collected post-intervention, 6,179 (or 40%) are attributable to the intervention.
In sub-group ITSAs, each group experienced an increase in VL coverage overall, although the increases were not uniform over the dimensions analyzed (Table 2). Compared to secondary health facilities, primary health facilities experienced a bigger increase in both the step and trend in VL coverage. On the other hand, Private and CHAM facilities registered a smaller increase in the step of VL coverage but a bigger trend than public health facilities.
Robustness checks and sensitivity analyses
When we introduced pseudo-intervention periods in the 6th and 12th weeks, we found that the model in equation 1 was robust. In the 6th week, the step change in VL coverage was modest at -16 percentage points and not significant (95% CI: -33, 1.9, P>0.05); similar results were obtained in the 12th week with a step change of four percentage points (CI: -19, 27, P>0.005). Thus, the big and significant step-change observed in the 18th week of the study (or the week in which the intervention was introduced) is attributable to the intervention.
In sensitivity analyses, the results remained robust. In the first sensitivity analysis, where the post-intervention period did not overlap with the new policy on VL testing milestones, the intervention had a bigger and significant immediate increase in the level of VL coverage but a non-significant declining post-intervention trend (step change=60 percentage points, [95% CI: 43, 77], P<0.001; trend=-0.21 percentage points per week, [95% CI: -1.88, 1.46], P>0.05); this analysis included 22 facilities. In the second sensitivity analysis, we removed Chiunda and Nankhwali Health Centers because they had outlying VL coverage; Chiunda had post-intervention VL coverage of 240% while Nankhwali had 149% pre-intervention coverage. However, after removing the two outlying facilities from the analysis, the effect of the remote WhatsApp supervision on VL coverage remained relatively unchanged.
Gaps in the quality of care received by PLHIV, particularly in resource-limited African settings where the majority of people with HIV live, call for low-cost and easily scalable interventions that can increase the uptake of HIV services. VL monitoring is one critical HIV service recommended by the WHO for monitoring response to ART, particularly in resource-limited settings.2 However, VL testing in these settings remains suboptimal and needs to be improved. We examined whether real-time remote supervision delivered through WhatsApp―a user-friendly and free technological platform—can improve VL testing coverage in rural Malawi. The WhatsApp supervision increased both the level and trend of VL testing coverage, with nearly half of all VL samples collected in the post-intervention period attributable to the intervention. Nearly all facilities in the district reported an increase in VL coverage. In sub-group analyses, the increase in VL testing coverage due to the WhatsApp supervision was greater among primary and public health facilities compared to secondary and private health facilities. In sensitivity analyses, the findings were robust across a variety of assumptions.
The results highlight two important things: First, the results show that remote supervision of the VL testing process via mobile instant messaging can improve VL testing coverage. Second, the results show that any programmatic changes, even if well-thought-out, may not yield desired effects unless they are accompanied by supervision. In the current study, prior program efforts aimed to actively identify clients due for a VL test, improve client flow, and increase the capacity to conduct VL tests did not improve VL testing coverage until real-time data reporting and rapid feedback via WhatsApp, which helped to identify gaps and act on them quickly, was introduced. Therefore, although it is plausible to argue that the effect reported in the current study is due to both the WhatsApp supervision and the programmatic changes, we believe that it is the real-time supervision that made the difference; for nearly one year, the would-be program changes did not improve VL coverage in the district.
As we highlight the effects of WhatsApp supervision, we acknowledge that mobile instant messaging can disrupt one’s workflow since the group members were responsible for many other things, not just conducting the VL tests or supervising the VL testing process. Therefore, the true effect of the intervention might be lower to the extent that the disruptions negatively affected staff performance in other areas supported by Baylor-Malawi―for example, ART adherence counseling. However, this may not have happened and the opposite may hold since VL testing is connected to other services such that improvements in VL testing coverage can have downstream positive effects, for example, on the number receiving intensive ART adherence counseling, as well as the number of clients being switched to other ART regimens.
The finding that the effect of the WhatsApp supervision was greater in public and primary facilities, compared with private and secondary facilities, is not surprising. Private facilities in the district do charge a user fee of K500 (USD 0.67) to access HIV/AIDS services, open at 7 am and already have good supervision systems in place, while public facilities do not charge any fees and, as part of the intervention, opened at 6 am, and often less directly supervised. Thus, the margin for improvement using the WhatsApp supervision and the accompanying program changes was greater for public health facilities. As for facility type, the volume of clients on a particular day is much greater in secondary facilities, compared with primary facilities, and it is possible that, even with the WhatsApp supervision, some clients due for a VL test in the secondary facilities were still missed.
The current study contributes to a rich literature base which suggests that supervision in the delivery of healthcare services is effective, including in African settings, although note that our effect size is larger than in previous reports and that direct comparison with this literature is somewhat limited. This literature examines the effectiveness of supervision in monitoring the VL testing cascade17 and the delivery of ART services,46,47 services that aim to prevent mother-to-child transmission of HIV,48 eye care,49 management of childhood illnesses,50 and antenatal care.51 Comparisons are limited because some of these studies used direct supervision or nurse/clinical mentorship,46,48,50,51 provided the supervision in non-clinical settings,47 or were unclear about the mode and delivery of the supervision.17 A closely related study done in Swaziland reported that supervision helped to improve the timeliness of communicating VL results to clients and switching those eligible to second-line regimens17; however, the supervision was only part of broader support rendered by Medicines Sans Frontier, its effect was not quantified, and was only raised as a potential explanation for improved outcomes. Additionally, the study was unclear on how the supervision was delivered and what supervision methods were used. In Uganda, supportive supervision of Community Health Workers increased the number of malnourished children receiving nutritional services and improved the cure rate and overall quality of care of these children.52 In Tanzania, Kenya, and Malawi enhanced supervision of primary health care workers only improved the delivery of eye care modestly and the improvements were not clinically significant.49 As the authors pointed out, high turnover and staff absenteeism who received the intervention likely attenuated the impact of the supervision.
Findings from the current study also contribute to a rapidly emerging literature on the use of mobile instant messaging platforms such as WhatsApp for communication and supervision, which suggests that the platforms are effective and can be a reliable tool for quality improvement in health and healthcare.53 While the supervision was the main intervention in the current study, we believe that the mode of delivery―the WhatsApp platform―re-enforced the effect of the supervision given that WhatsApp is easy to use, quicker, and allows photo-sharing.54 In this study, the supervisees did not have to enter daily aggregate clinic data in an electronic database, rather they just had to send aggregated data from manual registers at the sites and share with the Tingathe senior staff, which was not overly burdensome to them. Indeed, in Kenya, a study reported that nearly a quarter of WhatsApp communication between Community Health Workers and their supervisors contained photos, which helped to improve data documentation and teamwork55; the Community Health Workers also reported having positive experiences with the platform.55 In the current study, the WhatsApp groups flattened the hierarchy within Baylor-Malawi which made communications and participation in conversations between the sites and the supervisors easier and quicker thereby re-enforcing the effects of the supervision.54
To achieve durable viral suppression, the 3rd and final goal of the UNAIDS 95:95:95, finding low-cost interventions that can increase VL testing coverage and inform us of the progress towards that goal, is critical. This investigation shows that remote supervision delivered via communication technologies such as WhatsApp platform can improve VL testing coverage, which suggests that solutions to the problems facing the delivery of healthcare services in many African settings—particularly where logistical and geographical challenges may hinder service delivery—may be within reach. These findings are particularly relevant within the context of the COVID-19 pandemic where direct supervision has been curtailed.
Following these findings, Baylor-Malawi’s Tingathe program scaled-up the use of the WhatsApp platform to supervise many aspects of health services delivery in Malawi including monitoring of retention in HIV care, utilization of optimized ART regimens, ART multi-month scripting, staffing, stock levels, as well as facility response to COVID-19. We acknowledge that Baylor-Malawi was able to do this because many healthcare workers in Malawi have smartphones and already use the WhatsApp platform for communication, but this is also the case for other countries in sub-Saharan Africa.31 Thus, remote supervision via WhatsApp should be scalable not only in Malawi but within the region since all that is needed is a smartphone and internet connectivity. Further research may seek to understand the acceptability of using instant messaging platforms among healthcare workers in the wake of increased demands for data to be transmitted via these platforms.
Acknowledgments: We thank the Malawi Ministry of Health for their partnership in this work. We thank the Baylor College of Medicine Children’s Foundation Malawi site supervisors, the District HIV Programs Officer for Mangochi, clinical mentors who made the remote supervision possible and provided data on viral load samples drawn. This publication was made possible by support from USAID. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of USAID.
Funding: The Tingathe program is financially supported by USAID/PEPFAR.
Authorship contributions: SM, SD, CC, CP, TT, and MK conceived and designed the study, helped analyze the data, interpreted findings, and wrote the manuscript. SD and BR assisted with the study in the field, contributed to data management, and reviewed the manuscript. SM, MK, and SA critically revised the manuscript for contemporaneity and important intellectual content. All authors have read and approved the final manuscript.
Competing interests: The authors completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author), and declare no conflicts of interest.
Steven P. Masiano, Ph.D.
Baylor College of Medicine Children’s Foundation Malawi
Private Bag B397
Lilongwe 3, Malawi