The Expanded Programme on Immunization in the Low-and Middle-Income Countries (LMICs), including Pakistan, aim to achieve Fully Immunized Child (FIC) through having 85% of the target population of the one who had received one dose of BCG, of Penta (including deptheria-tetanus-pertusis in addition to Influenza b and Hepatitis B vaccines) I,II,III; OPV I,II,III; and measles-1 by the age of one year.1,2 Electronic Health (eHealth) refers to the practice and strategy to support health care through communication and information technologies. World Health Organization (WHO) considers eHealth initiative as a tool to strengthen health systems, improve health services delivery, and ensure access to those services.1 Mobile health (mHealth) is a subset, and it applies when mobile phones or any portable electronic devices with software applications are used for health interventions. The mHealth interventions have been applied in several LMICs to ensure access to health services and to improve health status through providing health related information, learning resources, reminders both to health workers, patients and communities.3–8 Use of mHealth in improving vaccination coverage has been in evolution, and there have some efforts in piloting or replicating evidence from within country’s or regional experiences even in Pakistan and neighbouring countries.9,10
Immunization is one of the most cost-effective interventions to prevent deaths from vaccine preventable diseases (VPDs), especially in children.11 While Pakistan has an overall routine immunization coverage of 66 per cent for fully immunized children, coverage in the province of Balochistan is much lower at 29 per cent.12 This is due to several barriers including challenging geography, low literacy and a decreased demand for immunization.13,14 Conventional methods, such as passive follow-up or expecting parents that they would show-up by themselves for children’s vaccination, have failed to improve this situation. Balochistan is the largest province of Pakistan by area and the least developed. At the same time, it suffers from several political, tribal and border conflicts. The distances to health facilities for the catchment population are long with limited accessibility, and the facilities have little outreach. The female literacy rate and knowledge about immunization is low in Balochistan (e.g., literacy just about 8 per cent),13 which may have made it difficult to for women in the province to develop a thorough understanding of immunization and the importance of herd immunity in communities.15 Moreover, low awareness levels amongst healthcare providers, concern about vaccine safety and beliefs in local remedies are major demand-side barriers to immunization.15 Lack of timely information and low confidence in the quality of services are additional demand-side barriers.12,16 Outreach services are mainly offered by Lady Health Workers (LHWs) and there are only a limited number of vaccinators for immunizing pregnant mothers and children.
The mHealth interventions are not limited to vaccination only. The technology has been effectively used reducing perinatal mortality rate, increasing exclusive breast feeding, for nutrition, health behaviours, patient medication compliance, maternal education and heath behaviour, and in strengthening health systems. Most of those interventions were implemented and tested among the LMICs of Africa and Asia, including Asian Subcontinent.5,7,17–19
Literature shows that several supply and demand side interventions help to improve immunization coverage.2,14,15,20–25 Certain demand side interventions such as immunization campaigns, monetary incentives, information transfers have been successful to increase the demand for vaccines and to improve vaccination coverage among children in several LMICs15,26,27 Various mHealth related interventions have been implemented to increase the uptake of vaccination by increasing awareness and providing relevant information about the time and places to get vaccines through reminders.1,3–8,17,19,24,28,29 Although mHealth interventions have potential to improve vaccination coverage by increasing demand, the evidence for such interventions is not adequate in Pakistan, and there is a lack of local robust research which could produce good evidence and inform the policy makers about the innovative solutions to improve childhood immunization uptake by the community. Considering this gap, this implementation research aimed to assess the effectiveness of introducing an mHealth intervention using an Artificial Intelligence (AI) platform based on Short-message service (SMS) and Integrated Voice Recordings (IVR) to remind and persuade parents to get their children vaccinated. Successful implementation of the innovation was hypothesized to 1) reduce delay in initiation of vaccination; 2) reduce the delay in subsequent vaccinations and 3) increase fully immunized child (FIC) achievement. This implementation research, particularly, looked at the implementation of Artificial Intelligence-based mHealth technology (SMS and IVR) in immunization in Quetta City in Balochistan province. It assessed acceptability and compliance with the new technology; and determined the effectiveness of the intervention on immunization coverage and dropout rates.
METHODS
It was a mixed method study with intervention. Data were collected in Quetta City through a mixture of qualitative and quantitative approaches. Baseline data were collected from 1,600 eligible parents within the catchment areas of 75 basic health units (BHUs) in Quetta. We used Lot Quality Assurance Sampling methods (LAQS) used for maternal and child health surveys for random selection of 1600 eligible (having a cell phone) participants (parents) for quantitative part of the survey. The same technique was used in both the baseline and the end line parts of the study. The primary outcome to be measured was any percent increase in vaccination in the target population.
Intervention
An automated platform was instituted with SMS and IVRs initiated by the Expanded Programme on Immunization (EPI) Quetta. The messages were developed in Urdu language (universally understood in Balochistan) by the experts in their field. They were also pilot-tested to understand if they were easily understood by the receiver. A unique number (6007) was used to send out the IVR and SMSs to the Cell phones of the parents. There were three IVRs developed: (1) IVR to ask parents of they had got their child vaccinated for the due vaccine in the immunization schedule, (2) a public service message to persuade vaccination, and (3) an IVR to the vaccinator reminding him/her of dropouts in their area of responsibility. Daily reminders and IVRs were sent to the Cell phone numbers of all the mothers/fathers listed in the database/records. Responses were noted on the AI platform. Asking if they had had vaccinated their children, on options of “yes” or “no”, the AI would generate appropriate response i.e., thank you for a “yes”, and sending the corresponding IVR to a vaccinator to approach them in case of a “no”. After three days, the same parent was again approached by the IVR for a “yes” or “no” response. Baseline data of 1600 eligible mothers/parents within the catchment areas of 75 BHUs of Quetta were collected, and the AI platform was instituted with SMS and IVR from “EPI Quetta” and 6007 starting November 2017 as there were technical difficulties in initiating the system. The daily reminders and IVR were sent to the Cell numbers in the records. Responses were noted on the AI platform. After a period of about two months, an end line survey of 1203 participants was performed with a loss of 397 participants due to their seasonal migration to warmer cities of Sindh.
Qualitative data collection and analysis
After a period of about two months, an end line survey of 1,203 participants was undertaken with a loss of 397 participants due to seasonal migration to warmer cities in Sindh province. In addition to the quantitative part, qualitative data collection as also conducted. Our objectives, to conduct the KIIs and the FGDs, were to explore barriers to the immunization and usefulness of mHealth, respectively. The question and guidelines for qualitative data collection were prepared, and piloted with three homogenous groups in an area not included in the survey. The questions were refined and adjusted accordingly. For the qualitative part, qualitative data were collected by data collectors who are experienced in conducting KIIs and FGDs and three key informant interviews (KII) were also conducted; including two KIIs with Lady Health Supervisors (LHSs) and one KII with the WHO staff responsible for routine immunization, were selected purposively to explore the barriers to immunization and usefulness of the mHealth initiative. In addition, four focus group discussions (FGDs), containing 6-10 mothers with same literacy level, were conducted to complement the data coming from Key Informant Interviews (KIIs). Out of four, three FGDs were conducted with caregivers (mothers with a child) and one focus group with respondents from an ethnic minority from within a highly educated community. The FGDs and KIIs were conducted in national and local languages easily comprehended by the participants. Respondents were asked about their perception about immunization programme and their experiences regarding mHealth intervention. For both the KIIs and FGDs, field-tested guidelines were used. The interviews were recorded with prior permission, and themes were developed for further analysis. Qualitative data were analysed by two authors and thematic analysis was done by developing codes and relevant themes. The main purpose of the FGDs was to explore the perception and practices of mothers regarding vaccination, usefulness of mHealth, SMS/call reminders, and barriers to timely vaccination. Analysis of the focus group discussions data yielded four main themes (Table 1).
Quantitative data collection and analysis
The questionnaire for quantitative data collection had five parts: consent form, household information, sociodemographic characteristics, immunization coverage form for infant immunization, and perceptions about childhood vaccination (based on Health Belief Model). The questionnaire was piloted for reliability and subsequent modification in the questions, accordingly. Quantitative data were collected by trained data collectors using notes from interviews and e-recordings with prior consent from the participants. In case if later it was realized that some responses were missing, the data collector tried approaching the respondent again. If a household refused to take part in providing data, or data remained incomplete, another household was randomly selected for data collection. The entire data collection process was supervised by two of the authors. Quantitative data were analysed to determine descriptive results (frequency distribution), for socio-demographic variables and for demand variables. Inferential statistics (t-test) was used to find any change from baseline to end line results. We also analysed for the prospective data of the unimmunized/immunized children through the intervening AI platform (IVR messages sent to the parents’ Cell phones and responses recorded). Data were collected, cleaned, coded and entered. Double entry in SPSS ensured correctness. All quantitative data and qualitative data (recording, notes) were kept confidential and in the possession of the principal investigator.
Ethics considerations
As this study involved personal information, we abided by the universal standards of ethical research. The Institutional Review Board (IRB) of the Health Services Academy was approached for the ethical permission for this research, which was obtained. We obtained written informed consent from all the participants and all records were kept confidential. There was no risk of harm to participants who participated in this research.
RESULTS
We randomly selected 1,600 children who were eligible according to the criteria among which 1,539 (96.2 per cent) were less than two years of age and 61 (3.8 per cent) were two or more years of age (see Table 2). Most of the respondents (both father and mothers of the children selected) were either illiterate or had studied up to the secondary level. About half of the respondents were employed (49 per cent), and about one third (37.9 per cent) owned a business. Some respondents (13.1 per cent) were unemployed. Regarding the total household income of the respondents, most of them had an income of less than 30,000 Pakistani rupees (US$300) per month. Very few (8.8 per cent) had an income above 30,000 Pakistani rupees per month. Mobile handset ownership was high, as 67.8 per cent owned a basic handset and 30.8 per cent owned android mobile phones (Table 2).
Age-wise distribution for immunization
A majority of study respondents (91.6 per cent) had an immunization card. Most of the respondents identified a BHU as the place for vaccinating their children. Bacillus Calmette-Guerin (BCG) vaccination coverage was highest (92.2 per cent) followed by oral polio vaccine (OPV) 0, 1, 2, 3 at 92 per cent, 86.1 per cent, 72.per cent, and 59.8 per cent respectively. More than half (59.1 per cent) of children had received an inactivated polio vaccine (IPV). Pentavalent 1, 2, 3 had been received by 86 per cent, 73 per cent and 60 per cent respectively. Pneumococal vaccination also followed the similar pattern of having Pneumo-1 being the most frequently received within all the three doses. However, at 9 months of age, only 25.2 per cent children were vaccinated for measles-1 and for measles-2, only 0.43 per cent of the children were vaccinated at the age of 15 months. A fully immunized child (FIC) is defined as the one who had received one dose of BCG, of Penta I,II,III; OPV I,II,III; and measles-1. For this, 398 (24.9 per cent) children were found to be fully immunized as per the definition stated earlier. Whereas, 1,202 (75.1 per cent) of the children were not fully immunized (Table 3).
Measles remains the least vaccinated disease in the study, with the first dose (measles-1) received by only 25.2 per cent and the second dose (measles-2) almost negligible, with only 0.5 per cent of the surveyed children vaccinated. For about 400 (25 per cent) of the children surveyed, the source of the vaccination for measles-1 was BHUs, but for measles-2, the source of vaccination was only 7 (0.4 per cent) for BHUs. No sources were reported for measles-1 and measles-2 among 1,196 (74.8 per cent) and 1,589 (99.3 per cent) of the children, respectively. Only 458 (28.6 per cent) of the children were found to be fully immunized before the age of one.
Artificial intelligence intervention reduces delay in following the vaccination schedule
Findings showed a percent increase in coverage for pentavalent, OPV and pneumococcal vaccines for first, second and third doses/boosters from the baseline to the end line data. This reflects that the delay in following vaccinations was reduced by the AI intervention (Figure 1).
Reduction in subsequent vaccination and improvement in FIC status
Following use of the AI platform, there was an increase in achieving full immunization coverage for both measles-1 and measles-2. The increase in measles-2 vaccination rates were highly encouraging, with an improvement from 0.43 per cent to 22 per cent among the sampled population. Delays in initial vaccinations and delays of subsequent vaccinations were reduced. The AI platform reminded parents about vaccinations even if they had forgotten the vaccination date or lost the vaccination card. It was also found that the AI platform facilitated sharing responsibility of vaccinations between both parents (Table 4).
Fully immunized child (FIC) achievement trend
A fully immunized child is defined as the one who had received one dose of BCG, of Penta I,II,III; OPV I,II,III; and measles-1. There was a marked improvement in the FIC attainment following the intervention (Figure 1)(Table 4). Furthermore, second dose of measles was markedly higher after the intervention.
Response of the caregivers to IVR
Only for the IVR purpose, out of 1,600 respondents of the baseline survey, 1,314 were found eligible for the intervention of IVR after removing 286 duplicates numbers. The 1,314, received messages from EPI Quetta. Out of the 1,314, only 572 (43.5% per cent) responded to the IVR (Figure 2), and (Figure 3).
Out of those who did not respond (742), 64 per cent did not pick up the call, 29 per cent did not choose any option and 7 per cent were busy tones (Figure 4). Some of the main reasons for lack of response, when explored during FGDs, were the low literacy rate of mothers; mobile phones being with their husbands, which rang when their husbands were at work; and not knowing how to respond further.
Out of those who responded, most (80 per cent) responded mentioned that they had their child vaccinated on the due date (see Figure 1). This aspect was explored during key informant interviews and the FGDs, and it found that the timely awareness messages and IVR had persuaded parents to bring their children to the health centre for the due vaccinations.
Results of key informant interviews and focus group discussions
The interviews and FDGs in Balochistan identified barriers and enablers to immunization. One main reason mentioned by participants was the extremely low level of awareness of immunization. Other barriers to immunization mentioned were distance to the health centres, transportation, gender discrimination, and non-trained vaccinators that led to fears of about vaccine safety.
One LHS said,
"parents find EPI centres far from their reach, they have problems with transportation, lose their vaccination cards, and are less educated, especially the mother. Even if they reach the EPI centre, most of the vaccinators are not well trained. Wrong injections do occur resulting in injury and disability to the child. Myths and religious beliefs are also among the barriers." She further added “IVR would be better than SMS in motivating and educating community, and if it were in local languages, it would have been more acceptable. Nonetheless, the intervention has visible impact on an increase and timely vaccination of the children in communities, more being among the highly educated subgroups of the populations”.
One of the WHO officers highlighted the importance of mHealth, and its demand creation and acceptability:
“We need to create the demand of mHealth in the government department s well as in the community. We have already proposed “e-vac” system of the Punjab government but there are issues of not having enough resources. An affordable technology, for poor province such as Balochistan, might be promising.”
Regarding the SMS and calls, the majority of the participants found them useful reminders for the timing of vaccination for their children. According to the mothers, registered contact numbers of mobile phones were usually with the fathers and hence that way they got timely informed about vaccinating the child. In some instances, even if the father was working outside or was not available, he could later inform the mother or the family to get the child vaccinated at the due date and time. This was considered a big advantage of using AI in getting children vaccinated. Some mothers suggested that the importance of getting vaccinations could be increased by making the vaccination card as document needed for eligibility for school admission or for applying for passports in the country.
Generally, participants viewed the use of AI as beneficial in ensuring that children get all the needed vaccines. Some of the mothers were annoyed with the repeated calls and suggested that calls should be on the date of immunization only. Participants suggested that the AI should be further developed with inclusion of child’s photo as a proof of identity, followed by SMS confirmation and data updated in the electronic records.
DISCUSSION
Acceptability of the AI intervention
The community, especially the mothers, appreciated the new approach of sending reminders for vaccinating their children. There was a high level of acceptance and desire to continue using the SMS and IVR services. One possible reason is that during the last decade ownership of a handset among the populations has increased enormously, particularly among LMICs. This has provided an opportunity to the health professionals to reach out to communities, target groups and vulnerable populations with ease and effectiveness.30 In line with the results of this study, several other studies demonstrated that mHealth interventions have high acceptability by the caregivers as well as the health workers in multiple settings.5,7,17,29,31–33 The intervention was equally accepted by the caregivers with low as well as high literacy rate and among different socio-economic population.
Effectiveness of AI intervention
Reduced delay in initiation of vaccinations
It is well-documented from a number of countries that delays in initiation of vaccinations is very common. The DPT3 vaccine has been found to be delayed by almost 11 weeks in LMICs. This delay is more pronounced for newly introduced vaccines.34 In our study, despite BCG and OPV0 usually having good coverage (above 80 per cent) in most of the cases, it was encouraging that the intervention improved this to 99 per cent coverage. Evidence showed that the booster dose on the designated date was also improved, increasing coverage of second and third boosters of pentavalent, OPV and pneumococcal vaccines. The results show that uptake of both the first and second doses of vaccines increased remarkably after the intervention, especially for the second dose of measles. This is significant because measles compliance remains compromised and delays lead to gaps in immunity and frequent outbreaks of the vaccine preventable diseases. Different approaches and integration of efforts and programmes with innovative solutions are frequently recommended in the literature for LMICs. Use of mHealth technology to initially bypass and then to support the health systems in LMICs, has resulted in decreasing delays in service delivery. Many studies have documented shortening the delays in accessing healthcare and health services, both for curative and preventive medicine. Systematic reviews conducted for effectiveness of mHealth in improving immunization in infants and for maternal health3,5,9,25,28,35
Reduced delay in following the vaccination schedule
We found that most of the mothers did have the immunization cards with them but only a few had fully vaccinated children in baseline survey showing non-compliance with the printed immunization schedule. Missing the initial dose of an antigen typically results in missing booster or next doses of the same antigen, and its concomitantly administered vaccines.36–38 Following the timelines for vaccinations is imperative to maintain good herd immunity in a population. Other studies from different settings showed that differences in achieving the next date of vaccination has implications for infant and childhood morbidity and mortality.34,39 There is a limit to delay (up to two weeks to a month) and beyond that immunity cannot be boosted appropriately, and a child becomes susceptible to disease.40 Parents’ education and awareness levels, distance from the facility, vaccine stocks, vaccinators’ education levels, and location of birth can potentially contribute to delays in timely immunization.2,38,41
Findings of the study explored whether delay for the following vaccinations was shorten after introduction of the AI intervention. By comparing the vaccination percentages for each antigen between baseline data and the end line data, and understanding from the KIIs and the FGDs, it was evident that the delays for the following vaccination dates were reduced markedly. Reminders through SMS and calls helped to sensitize and inform parents about vaccination and vaccination schedule which ultimately increased their demand for, and uptake of vaccines. In many cases, fathers were the owner of the mobile phone and mothers did not have the access to it. However, despite being a patriarchal society, fathers showed a positive attitude and delivered the information related to vaccination to their wives and supported them to uptake the required vaccines for their children. Similar experiences were documented in other LMICs which showed that SMS reminders have resulted in a decrease in dropout rates and reduced delays in vaccination schedules.8 Ethnic communities from developed nations have also used simple tools, such as calendars, to improve the vaccination coverage effectively.38
Fully immunized child achievement trend
Vaccinations on time lead to achievement of a fully immunized child (FIC) among children under two years of age. Children from developed as well as from developing countries fail to achieve this status due to many factors as explained previously. Inadvertent delays between DPT1 and DPT3, and measles lead to typical lag and delay in achieving FIC status among children.2,38,40,41 Receiving measles vaccinations along with BCG, OPV I, II, III, Penta I, II,I II, and Pneumo I, II, III are considered to constitute FIC as defined by WHO. The comparative results showed an upward trend in the uptake of all the antigens for FIC, especially for third doses and measles-1 and 2. Barriers to FIC include parental education, parental attitude and knowledge, failure of the immunization system, and lack of proper communication and information. Several literature showed that timely information and communication can improve vaccination by one dose which are in line with the findings of this research.20,21 Awareness and reminders through mHealth initiatives help in sensitizing and raising awareness among the caregivers about vaccine, its availability and providing timely reminders on when they are due which positively influence in improving FIC coverage. However, few studies showed that mHealth interventions did not have much impact on the improvement of the FIC coverage. A study conducted in 2014 in Karachi showed a low response rate to SMS and reminders. However, the situation has improved over the period as the format and language of the messages were improved.26
Increased demand to expand AI intervention
Given that about 100% of the world population are within mobile network several mHealth interventions have been designed to leverage mobile phone penetration in LMICs to improve healthcare access and dissemination of health-related information. Many of the interventions have been successful and demonstrated an improved vaccination coverage in rural hard-to-reach areas.6 Results of this study showed that health professionals and implementers suggested for scaling-up of the AI intervention. Participants in FGDs also highlighted the importance of expanding SMS and IVR based services to other areas in addition to immunization. It was because of the acceptability and feasibility of the intervention as it shown it other studies.3–6,17,25 However, further improvements leading to completely computerized vaccination card system, LHWs and vaccinators having children’s vaccination cards on their handsets, and regular reminders to parents on the due date were some of the recommendations from the major stakeholders of immunization programme.
Strengths and limitations of the study
The study population had both illiterate and educated mothers, which provided a broader understanding in the KIIs and FGDs. Mothers welcomed the SMS and IVR, and tried to comply accordingly. Data collection was as accurate as possible. A comparatively large sample size was also a strength.
There were challenges in the implementation of the intervention. The telecom service provider could not expand it to all cellular networks. Later in the project, a universal SMS portal was used for awareness outreach to all of the registered sample population. Targeting each child with specific SMS or IVR was not possible in the current set up of the AI, although the set up improved at the end of the project. There was also a loss of 28 children who died due to non-vaccine-related causes during the study. About 300 of more families left the intervention area during winter break to go to Karachi and other cities in Sindh where they usually go to meet their relatives and enjoy a less cold weather. Therefore, at the end line we had a smaller sample size of 1,203 participants instead of the 1,600-sample size we had for the baseline. One more limitation of our study was an absence of a comparison group, which however, was not predetermined for the current study.
CONCLUSIONS
This study showed marked improvement in the up-take of vaccination for the children following the introduction of the AI-based intervention. Discussions with the mothers and other stakeholders showed the intervention to be widely accepted and an important tool in increasing demand for vaccination in the community. The intervention has the potential to become more technically sound and user friendly. The potential for using technology to improve coverage is promising. However, few suggestions to consider for : 1) AI-based intervention has the potential to improve immunization coverage and equity in immunization, and to generate demand and address vaccine-related vaccine hesitancy, the technology should be scaled up in Balochistan and subsequently across Pakistan; 2) As the AI-based intervention is implemented and scaled up, lessons learned should be well documented to allow other locations to benefit from the initiative; and 3) The technology should be developed based on working with the community to meet the needs expressed at the local level.
Acknowledgements
We acknowledge the Alliance for Health WHO, the UNICEF, the GAVI and the Health Services Academy Islamabad for providing grant to conduct this research. We also acknowledge Faraz Khalid, Alyssa Sharkey and Saadia Farrukh from the UNICEF for their continued support, assistance and feedback during the development of this article.
Ethics and consent
Health Services Academy’s Institutional Review Board (IRB) approved the study.
Funding
Grant by Alliance for Health Policy and System Research, World Health Organization, Geneva, UNICEF and GAVI, the Vaccine Alliance.
Authorship contributions
EAK and MIP and BS contributed to the conception, design and drafting of the work; EAK, AS and SA revised it critically for important intellectual content and approved the version to be published.
Competing interests
The authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no competing interests.
Correspondence to:
Ejaz Ahmad Khan, MBBS, MPH, FFPH. Associate Professor Health Services Academy, Prime Minister’s National Health Complex, Park Road, Chak Shahzad, Islamabad, Pakistan. Email. [email protected]