Full routine immunization for children below 24 months now costs between US$37 and US$101 across different settings in low-and middleincome countries as more and costlier vaccines are being introduced into national immunization programs [1]. The rising cost of vaccine and vaccine distribution, global supply shortages, and the introduction of new vaccines have provided the impetus to deploy a real-time management system to negate the uneven distribution of vaccines, improve efficiencies and allow better visibility of the vaccine supply chain [2,3].

A robust vaccine cold chain and logistics system including storage and transportation is the London Journal of Medical and Health Research cornerstone of all immunization programmes [4]. Rigorous supply planning based on accurate data should enable such a system to have the right vaccines in the right place, at the right time, in the right quantities, in the right condition, and at the right cost [3].

The 2021 Multiple Indicator Cluster Survey-National Immunization Coverage Survey (MICS-NICS) report showed that only about 44.2% of all children between 12-23 months old have received all basic vaccination doses in Nigeria; while only about 40% of those aged between 24-35 months had received all the recommended vaccines on the national immunization schedule in Nigeria in 2021 [5].

These indicators for measuring RI performance have shown slow progress in full immunization coverage in Nigeria despite huge investments and improvements in supply chain and distribution of vaccines in the last 5 years. However, the increasing incidence of stockouts and disrupted access to vaccines is of growing concern [3]. The majority of these stockouts are a result of internal issues such as inaccurate forecasts, stock management issues and restock delays [6].

Vaccine supply chain management systems have been prominent features of the Expanded Programme on Immunization (EPI) plans from the onset [7]. System requirements have expanded dramatically over the past several years with the introduction of new vaccines and the frequent mass campaigns to control, eliminate, or eradicate specific diseases, such as polio, measles, rubella, tetanus. While such initiatives tend to be well-funded, the strain they place on the supply chain system are substantial and often not acknowledged, particularly at state levels [7]. The most visible impact of new vaccine introduction is an increase in the volume of products that need to be stored, transported, and tracked, as well as the need for more storage capacity due to the increased use of single-dose vials [8].

These further exacerbates existing and new challenges in vaccine supply chain management like poor cold chain equipment inventory and status; poor vaccine stock, distribution and utilization data collection and management; lack of skilled and properly trained workers in vaccine management operations; poor vaccine storage facility; and unreliable micro-plan data which leads to wastage, stock-outs and overstock, and inequitable distribution of products [9,10,11]. It is important to note these challenges and many other vaccine supply chain management problems are related to inefficient database management systems and innovative digital tools [9,12].

In 2016, the Ministerial Conference on Immunization in Africa laid the groundwork for the landmark Addis Declaration on Immunization (ADI), including 10 commitments to achieve universal and equitable access to immunization in Africa. The ADI was endorsed by Heads of State from across Africa at the 28th African Union Summit in early 2017, signaling political support for immunization on the continent at the highest possible level [13]. Real-time data and digital tools were identified as a powerful instrument to advance these efforts. Digital platforms make it easy to assess progress and to revisit decisions along the way, instead of waiting for quarterly reports or midline assessments. Digital tools also democratize learning and decision-making, so that stakeholders at all levels of the health system can make meaningful use of data.

In response to these assertions, eHealth Africa, a non-governmental organization (NGO) was funded by the Bill and Melinda Gates Foundation (BMGF) and the Kano State Primary Health Care Development Agencies (KSPHCDA) to conceptualize, design and roll out an electronic logistic management information system, LoMIS Suite to strengthen accountability and data-driven management in vaccine supply chain management in 2014. The Suite consists of an offlinecapable mobile application "LoMIS Stock'' and a web application and "LoMIS Deliver." LoMIS Stock was designed to help health care workers (HCWs) bypass the traditional paper-based reporting system and submit reports instantly using an app on their mobile device. The app was also built to enable supervisors to get near real-time visibility of stock inventory, consumption rate (vaccine wastage, stock out, utilization), cold chain status and cascade London Journal of Medical and Health Research deliveries from all locations where vaccines are stored. LoMIS Stock answers three critical questions "Where are my Vaccines? ""Are they sufficient?" and "Are they potent?" LoMIS Deliver on the other hand aggregate these reports on stock levels, equipment status, and vaccine utilizations into a single web dashboard for real-time data visualization of field operations.

Both applications are based on the structure and workflows of the KSPHCMB system.

This study was conducted to answer the following research questions:

1. Is the LoMIS stock tool being used by the proposed users to report routine immunization vaccine supplies? 2. What is the effect of its use on the duration in reporting stock out and the turnaround time? 3. What is the impact of the tool on facility stock out experiences? 4. How has the tool supported the routine immunization stakeholders to plan for vaccine distribution?

In line with the research questions, we were guided by the objective of understanding the impact of using digital data management tools in vaccine supply chain management against the traditional paper-based systems in Kano state. Specifically,

The main limitation encountered in this paper is the quality of data and data availability in the DHIS2 data. Especially in Kano state, we were unable to conduct robustness checks on stock out by vaccines due to the unavailability of data.

We employed the quasi-experimental design using a mix of quantitative and qualitative research methods to determine what would have been the outcomes in Kano state if the digital tool (intervention) had not been utilized. In this study, we identified Bauchi State vaccine supply chain management system as the comparison group based on validated evidence that Bauchi state was utilizing the paper-based reporting tools for vaccine stock inventory as of the period of this study.

Our study respondents were selected by using a three multi-stage sampling technique to select and recruit sixty (60) and forty (40) Ward Technical Officers/Routine Immunization Officers (10) for the intervention (Kano state) and the comparison (Bauchi state). We stratified all the state LGAs under the three senatorial zones in both Kano and Bauchi States and used a purposive sampling approach to select 6 LGAs under the 6 Administrative Zones of Routine Immunization in the Kano State, and 4 LGAs under the 2 Administrative Zones of Routine Immunization in Bauchi State based on their geographical proximity to each other. Ten (

Primary data for this study were collected using the evaluation tools (structured questionnaires and Key in-depth interviews) while secondary data was collected from records on the LoMIS Dashboard and District Health Information System Software (DHIS2). LoMIS dashboard is a platform that enables near-real-time visibility, storage, and retrieval of vaccine stock inventory while DHIS2 enables the collection, storage, retrieval and management of case-based data records. The primary data sources provided information on duration of reporting vaccines, duration of response to reports, and the benefits of LoMIS. The secondary data sources also provided information on the duration of vaccine reports and duration of response to those reports. The secondary data sources provided more information including the usability of LoMIS, availability of stock or stock-out.

In this study, it was found that males are the predominant staff working within the routine immunization areas across Kano (73%) and Bauchi (90%) states (Figure 4).

The findings related to the amount of time it takes to report RI vaccine inventories via LoMIS Stock in Kano state and the paper-based tools in Bauchi state were compared. In Kano state, it was found that 67% of respondents reported that it takes Pop more than 10 minutes to complete the process of reporting vaccine stock inventories via the paper-based tools while 100% of the respondents reported an average of 5 minutes to complete a report via LoMIS Stock. This is consistent with the findings from the comparison group (Bauchi State) where 63% of them reported spending more than ten minutes whilst reporting through paper-based tools.

Findings from the tool dashboard validates the duration of reporting via LoMIS. Users of LoMIS application tool used an average duration of less than 5 minutes on any stock inventory form as illustrated in Figure 6.

In Kano state, it was found that out of the 39 respondents that provided answer to the question of average response time to stock out reports, 34 (57%) reported that all stock-out reports done through the LoMIS tool got response under 24 hours as compared with 20% that got response under 24 hours prior the use of LoMIS stock.

Twenty-two (59%) of 37 responses revealed that Cold Chain Equipment breakdown through LoMIS got a response within 24 hours as compared to 10% who got a response in 24 hours before the use of LoMIS (Figure 7).

Source: Study survey data

Another major finding from this assessment study is reduction in stock-out experiences in health facilities. It was found that 60% of facilities had at least reported an event of stock-out from January experiencing stock-out reduced with time in Kano state. Within 3 years of using the LoMIS tool, the number of facilities experiencing stock out reduced by 92% in Kano (LoMIS) as compared to only 42% reduction in Bauchi (paper-reporting tool) within the same period (Figure 8). 8). In addition, the differences in stock out outcomes in Kano and Bauchi states were confirmed as the Analysis of Variance (ANOVA) showed that the effect of the reporting tool on stock-out was significant (F (1,70,) = 215.506, p = 0.000). The comparison analysis using a paired-sample t-test showed that the mean difference in facility stock-out between a 3-year period prior and after LoMIS was found to be statistically significant (p=0.000) (Table 1).

This implies that LoMIS significantly contributed to reduced experiences of vaccine stock-out in health facilities. The independent sample T-test also revealed a huge significant difference between the means of facility stock-out in Kano (use of LoMIS) and Bauchi (Paper-tool) with a large effect size. Again, this result further validates the contribution of LoMIS to reduced vaccine stock-out in Kano (Table 2). 9). This further validates the effect of LoMIS on reduced stock-out occurrences in Kano state.

We conducted a comprehensive assessment of the impact of LoMIS Stock reporting tool on vaccine stock management in Kano state. The results showed that LoMIS Stock has contributed to effective vaccine management in terms of real-time reports, vaccine visibility and accountability. These have led to timeliness of report, increased response rate, reduction in facilities experiencing stock-out and eventually availability of adequate stock at the last-mile facilities. The findings from this study further corroborates the results of several studies on the impact on the use of digital tools as an effective approach to public health interventions globally [15,16,17,18].

In Nigeria, one of the challenges of vaccine stock management occurs because of poor vaccine stock visibility at health facilities [19]. Our study has shown that this challenge can be significantly addressed with a digital tool as we found that the use of LoMIS Stock for reporting vaccine stock inventory enhanced access to vaccine stock distribution across the focal facilities. The usability rate (82%) of LoMIS Stock shows that the use of digital tools is a prioritized solution to the challenges of public health interventions in low-and middle-income countries like Nigeria. study where it was found that digital technology is a feasible and acceptable approach for public health interventions [20]. Specifically for immunization interventions, a recent study reported high acceptability rating amongst vaccinators and district managers for an iterative mobile-app that was developed to track immunization encounters in Pakistan which shares similar public health demographics with Nigeria [21]. In another study in Malawi, end-users assessed the usability of two digital tools for community case management of under-five children-highlighting that although both tools are useful in gathering important intervention data, their usability differs by a wide margin and this determined the overall success of the tool in supporting health interventions in the study setting [22]. This implies that the ease-of-use that was recorded with the end-users of the LOMIS Stock App contributed significantly to the success of the tool in managing vaccine supply chain management in Nigeria.

We found that the amount of time needed to complete a report of RI vaccine inventory significantly reduced using a digital health tool. This corroborates the findings of a digital health responses were consistent with pre-LoMIS findings in Kano State. In another study, the results also posits that the use of digital technology for reporting routine immunization data is feasible and can provide real-time updates to RI performance indicators such as vaccine availability [23] .

In addition, this study sought to assess the impact of the digital tool on facilities experiencing stock-out; and we found that the stock-out experiences reduced over time in Kano state.

Within 3 years of using the LoMIS tool, the facilities experiencing stock out reduced by 92% in Kano (LoMIS) as compared to only 42% reduction in Bauchi (paper-reporting tool) within the same period. The implication of this result is that there was sufficient stock of death-preventing vaccines at health facilities in the study location.

This finding strengthens the discussions around the value of developing and implementing digital health tools in public health interventions [24].

Furthermore, the impact of the tool, LoMIS Stock on vaccine planning and management was significant as our results showed that the LoMIS dashboard tool enhanced the vaccine operations visibility-providing near real-time data that enabled key decision makers to respond to stock-outs, CCE breakdown, and vaccine over-stock in record time, and ensure that immunization efforts meet expectations. Improved planning and management organization is another notable impact of the LOMIS dashboard on decision makers in the study location as we recorded in this study. This further contributes to existing evidence that data dashboards in public health tools provide reliable information to inform and support policy makers and project managers in iterating public health interventions [25]. Several sources show that a wide range of COVID-19 response interventions utilized many dashboards to collect data on time series, geographic maps, case incidents, contact tracing, community surveillance and clinical trials to refine intervention approach, outputs and outcomes [26,27,28,29]. A similar result was obtained by another researcher where a proactive infection prevention and control (IPC) monitoring tool was pioneered to provide reliable data for real-time response to emerging risks to COVID-19 infection amongst healthcare workers in Guangdong Second Provincial General Hospital in China. The study showed that the worrisome rate of healthcare workers infection prior to the deployment of the tool was reduced; and future plans to integrate artificial algorithms to the tool for improved outcomes are already being muted [30].