Structured questionnaires were developed using the Hydrocele Surgery Facility Assessment Tool (HSFAT) developed by LSTM (7) for hydrocele and a precursor to the WHO DIP for lymphedema (6).

The HFAT consists of the HSFAT facility-level questionnaire (7) and a newly developed patient level questionnaire (see Supplementary Material: Data sheet 1 for full tool). As described previously (7), the facility level questionnaire consists of questions and key (equally scored) indicators grouped across the following 10 domains: 1) background information, 2) essential amenities, 3) emergency patient transfer, 4) laboratory capacity, 5) surgical procedures and trained staff, 6) infection prevention, 7) non-disposable basic equipment, 8) disposable basic equipment, 9) essential medicines and 10) current hydrocoele practices ( Table 1 ). The patient level questionnaire covers: knowledge to make an informed decision, changes in economic situation following surgery, changes in family life following surgery and suggestions for improvements.

The LFAT corresponds to the WHO DIP with questions relating to the facility level questionnaire and a patient interview (see Supplementary Material: Data sheet 2 for full tool). The facility level questionnaire consists of questions and key (scored/tracer) indicators grouped across the following domains: 1) background information (not scored), 2) trained staff, 3) case management and education materials, 4) infrastructure, 5) medications and commodities, 6) patient tracking system, 7) staff knowledge, 8) challenges and feedback (not scored), 9) staff demonstration (not scored). Table 2 highlights some minor differences between the WHO DIP and the LFAT with indicator wording and scoring of indicators under domains 5 and 7. As much as possible, the format of the results presented in this paper is aligned with the WHO protocol. The patient interview assesses knowledge of strategies for preventing acute attacks and the progression of lymphedema, and collects patient feedback and suggestions on service improvement.

The scoring of the indicators in the HFAT and LFAT provided national, facility and indicator level scores. A score of 75% was set as a benchmark for health facilities and indicators as a minimum standard to achieve. The analysis herein presented focuses on the indicator level scores.

The assessments described in this manuscript were implemented between May 2022 and January 2023 in Côte d’Ivoire, Guinea-Bissau, Liberia and Mali. All the assessments took place in intervention areas supported by Sightsavers. In the case of the LFAT, 10% of health facilities in the targeted areas (in each country) were randomly selected. In the case of the HFAT, all hospitals declared able (by health authorities) to provide hydrocele surgery were assessed.

Tables 3 , 4 provide summary information on the facilities assessed in each country. For the purpose of this paper, health care facilities were categorised using standard health care facility classification in developing countries, including four distinct tiers: local health posts (tier 1), district health centres (tier 2), province- and regional-level hospitals (tier 3), and national and central hospitals (tier 4) (12).

In Côte d’Ivoire, the assessment area covered six health districts (Tiassale, Tiebissou, Divo, Yakasse Attobrou, Lokota, Bongouanou) across five regions (Agneby-Tiassa, Belier, Loh Djiboua, Me, Moronou), ( Figure 1 ). The HFAT was implemented in six hospitals (tiers 2-3) and the LFAT in 19 front line health facilities (tiers 1). Assessments were done from 18 to 23 October, 2022.

In Guinea-Bissau, the study targeted five regions (Bafata, Bijagos, Cacheu, Gabu, Quinara), 19 health areas (Bafata, Bambadinca, Contuboel, Sare Bacar, Bubaque, Caravela, Formosa, Uno, Barro, Canchungo, Carenque, Sao Domingos, Canjadude, Gabu, Pirada, Tumana, Brandao, Buba, Empada) ( Figure 2 ). The HFAT was implemented in five regional hospitals (tier 3) and the LFAT in 19 health facilities (tier 1 and tier 3). Assessments were done from 15 December, 2022 to 15 January, 2023.

In Liberia, the assessment was conducted in five counties (Bomi, Bong, Lota, Maryland, Nimba), 14 districts (Clay, Sinji, Jorquelleh, Suakoko, Foyah, Kolahun, Salayea, Voinjama, Zorzor, Harper, Karluway 2, Plebo, Saclepea Mahn, Tapeta) ( Figure 3 ). The HFAT was implemented in 11 hospitals (tiers 2-4). The LFAT assessed 26 health facilities (tiers 1 –3). Assessments were done from 14 to 21 July, 2022.

In Mali, the assessment targeted one region (Sikasso), ten districts (Bougouni, Kadiolo, Kignan, Kolondieba, Koutiala, Niena, Selingue, Sikasso, Yanfolila, Yorosso) ( Figure 4 ). The HFAT was implemented in one regional hospital (tier 3) and 10 referral health centres (tier 2). The LFAT assessed the regional hospital and 10 referral health centres (tiers 2-3) as well as 27 community health centres (tier 1). Assessments were done from 9 to 16 May, 2022.

The structured questionnaires were uploaded on the CommCare platform ² so that field teams could digitally collect data on structured, error-detecting forms; that data was immediately sent into protected cloud storage. A dashboard using Metabase ³ and a set of automated reports, accessible to designated health staff involved in the activity, were also developed to monitor the progress of the assessment and make reporting easier. They include the final score obtained by each health facility and for each indicator. The data was cleaned during data review meetings organised in each country at the end of the assessment period. To do so, data was downloaded from CommCare in the Excel format then cleaned and updated based on feedback provided by country teams. It was then reintegrated and used for scoring.

Training for all enumerators, LF programme supervisors, additional Ministry of Health staff, and Sightsavers country office staff was rigorously conducted to ensure all personnel had sufficient skills and experience. Training topics covered included the protocol, the CommCare application and data collection procedures, the evaluation instruments (questionnaire using CommCare), quality control methods and the online dashboard. Due to their length, the questionnaires and protocols were sent to all designated enumerators prior to the training to ensure that they were familiar with the tools.

Prior to implementing any assessments, authorisation to collect data in the area (region and/or district, depending on the country) was obtained. National LF programme coordinators then wrote letters to each health facility which would be assessed; the letters were presented to the health facility lead upon arrival. Assessments took anywhere between a half to a full day in each facility.

The structured questionnaires were administered by asking questions one after the next. One enumerator asked the questions while the second recorded the answers on the smartphone or tablet. Where demonstrations of certain activities were part of the assessment, the enumerators observed and noted the performed gestures and attitudes.

In the case of the LFAT, staff of medical wards/dermatology departments and front-line health facility doctors, nurses and other health personnel provided the answers. For the HFAT, surgeons and their staff are targeted to answer questions but if the surgical team could not answer a question support also came from hospital administration, laboratory, or pharmacy staff. Information on the number of beds and the availability of fuel could also be provided by the staff of the finance department; information on personnel could be provided by human resources managers or the director of the facility. In the operating theatres, depending on the availability of the staff at the time of the interview, the surgeon answered alone or was assisted by assistant surgeons and anaesthetists.

The facility benchmark of 75% was attained by 32% of the facilities assessed in Mali, 12% of the facilities assessed in Liberia and 0% of the facilities assessed in Cote D’Ivoire and Guinea Bissau. Across countries 15% of the facilities assessed attained the 75% benchmark.

Summary indicator results from the LFAT assessments are presented in Table 5 (‘Mean number of key indicators marked positively for each facility and overall percentage of positively scoring indicators by domain’). Figure 5 graphically represents the percentage of facilities that achieved quality indicator criteria for domains 2 to 7. A full breakdown of LFAT indicator results is provided in Supplementary Material: Table 1 .

The facility benchmark of 75% was attained by 100% of the facilities assessed in Liberia, 91% of the facilities assessed in Mali, 50% of the facilities assessed in Cote D’Ivoire and 40% of the facilities assessed in Guinea Bissau. Across countries 79% of the facilities assessed attained the 75% benchmark.

Summary indicator results from the HFAT assessments are presented in Table 6 (‘Overall percentage of positively scoring indicators by domain’). Figure 6 graphically represents the percentage of facilities that were marked positively on HFAT indicators under domains 2, 3, 4, 5, 6 and 10. The full table can be found in Supplementary Material: Table 3 .

Additional detail for domain 4 (four scored indicators), domain 7 (30 scored indicators), domain 8 (29 scored indicators), domain 9 (11 scored indicators) and domain 10 (seven scored indicators) is provided in Supplementary Material: Tables 4 – 6 . Across countries and domains the overall percentage of indicators scoring positively was, in descending order, Liberia (90.5%), Mali (85.3%), Côte d’Ivoire (71.9%) and Guinea-Bissau (64.8%).

Across the domains ‘Surgical procedures and training’ (domain 5) stands out as a particularly poorly performing domain in all countries except Liberia, where the percentage of positively scored indicators was 100%. By contrast the overall percentage of positively scored indicators was < 60% in Côte d’Ivoire, Guinea-Bissau and Mali. In descending order, the % of facilities routinely using a surgical safety checklist was Côte d’Ivoire (50.0%), Mali (27.7%) and Guinea-Bissau (0%).

A total of 21 patients were interviewed, 15 in Mali, four in Côte d’Ivoire and two in Liberia ( Table 7 ). All patients (100%) reported that they had received enough knowledge to make an informed decision about accepting hydrocele surgery. Overall satisfaction with service delivery was 81%. Among the 21 patients 52% reported improvement/significant improvement in socioeconomic status and 81% reported improvement/dramatic improvement in family life.

Previous DIP assessments conducted in Ghana (13) and Vietnam (14) for lymphedema management, and Bangladesh, Malawi, Nepal, Benin, Burkina Faso, Ghana, Guinea, Niger, and Nigeria for hydrocele services (7) demonstrated the important use of DIPs to document which facilities are offering services and yielded actionable suggestions to strengthen MMDP services and compliance to WHO’s recommended essential package of care. Data from these assessments will support individual country claims to have eliminated LF as a public health problem as part of dossier submission to WHO for validation.

This paper has described the process and approach of using DIP methodologies for both lymphedema and hydrocele in Côte d’Ivoire, Guinea-Bissau, Liberia and Mali through the facilitation of the electronic based LFAT and HFAT tools. The ability of these countries to successfully document results of the assessments in 10% of health facilities for lymphedema and 100% of hospitals targeted to provide hydrocele surgery will also assist the country programmes to achieve the dossier requirements set out by the World Health Organization for the elimination of LF.

Similar to other DIP assessments the results indicate that each country has unique areas for improvement and the data generated from the assessment can be used to develop a quality improvement plan, such that countries can be better assured that people affected by LF morbidity are being offered quality hydrocoele surgery and instruction on the management of their lymphedema. Because quality issues may compromise the availability of essential packages of care it is important that national LF programmes track completion of action steps that arise from the assessment. The potential use of these tools to assist in the maintenance of general health services to communities suggests that national programmes could apply these tools to other surgical services such a trichiasis surgeries and management of other skin diseases such as leprosy, yaws, Buruli ulcer, and scabies.

The results of the lymphedema facility assessments may be linked to prior experience in implementing lymphedema management at both the level of the country and the specific geographic area. As highlighted in the introduction, it was only in Liberia that all the facilities assessed had experience in implementing lymphedema management prior to the assessments. However, even in Liberia only 12% of the facilities assessed attained the 75% benchmark.

While the attainment of the 75% facility benchmark for the LFAT assessments was notably low in all countries, we acknowledge that the 75% figure (or any score) - as explained in the WHO DIP guidance - does not indicate pass/fail (6). Instead it indicates the considerable need to explore specific causes of underperformance and develop site specific corrective actions. The scores from these assessments may help country programmes set benchmarks for future assessments and track quality improvements over time.

With regard to indicator scores, the only indicator that did not achieve the 75% indicator benchmark in any of the countries was ‘trained staff - Community Health Care Providers trained in lymphoedema management’. The need for staff training was corroborated by both the interviews with facility staff on staff knowledge, challenges and service improvement suggestions, and the patient interviews. In the case of Mali, Guinea-Bissau and Côte d’Ivoire, related quality improvement actions led to the training of health workers, community health workers and patients. The issue of technical capacity and health care workforce is highlighted in the LF disease summary of the WHO 2021-2030 NTD Roadmap - with the recommendation to “include LF morbidity management modules in health workforce training curriculums” (15). Programmes may look to The Togo National Lymphedema Management Programme which achieved inclusion of lymphedema management in the routine healthcare package for learnings in this regard (integrating lymphedema management into the curriculum for medical staff was one of the five major components of the programme) (16). The evaluation by Dung et al. of lymphedema management services in Vietnam, two years prior to WHO validation (in 2018) illustrates how lack of trained staff is not just an issue for new lymphedema management programmes (14); their evaluation found that none of the facilities assessed had any staff who had been trained in the last two years.

In all countries except Liberia, scores on availability of MMDP guidelines were notably low. The low rates are due to the lack of implementation of morbidity management activities in the targeted areas of these three countries. With the exception of Côte d’Ivoire (with score of 100%), the results were similar for indicator 3.4 assessing the availability of local language/pictorial patient education/awareness. Development of Information, Education and Communication (IEC) materials/implementation of IEC activities was subsequently included as a corrective action in all four countries.

The availability of the named medicines was variable both within and between countries. The absence of ivermectin in any of the health facilities assessed in Mali was linked to the fact that LF mass drug administration (ivermectin + albendazole) has been stopped throughout the country; albendazole, which is also used as an intestinal dewormer, was available in 36.8% of health facilities. The cross-country issue with the availability of medications was corroborated by the interviews with health facility staff. The lack of medication/supplies was highlighted as top challenge to providing quality lymphedema care to patients. Whether availability of a named medication translates through to availability for a lymphedema patient may be questioned. Edmiston et al. reporting on the results of a lymphedema facility assessments in Ghana noted that “health facilities may feel as though LF patients should be treated with medicines and commodities that have been provided by the NTD Program, and do not consider those patients as part of the general patient population” (13).

The availability of the components of the washing kit was variable both within and between countries, and notably low in Côte d’Ivoire. Among the list of components, gloves scored relatively high as gloves are part of the items regularly available in health facilities. Subsequent to this assessment, partners supported the provision of washing kits (free of charge) to all patients within the assessment areas of all four countries.

As explained above, scores may be linked with prior implementation experience at both the national and sub-national level. In the case of the HFAT, Mali, the country with the most experience with offering hydrocele surgeries through the LF programme, had the highest overall score. The Mali LF programme started support for hydrocele surgery in 2008 with the National Programme for LF Elimination free hydrocele surgery campaign in Kolokani district (Koulikoro region). Five of the ten districts covered by the hydrocele facility assessments reported in this manuscript had received former support through the LF programme for the provision of hydrocele surgeries.

Across the nine scored domains, the results from domain 5 ‘surgical procedures and training’ were notably low (30.0-59.1%) in all countries, with the exception of Liberia (with a score of 100%). The results from this domain highlighted that the use of the WHO surgical safety checklist is not a widespread practice; in the case of Guinea-Bissau, none of the facilities assessed routinely used the surgical checklist.

The results around current hydrocele practice highlighted relatively good practice in Liberia (all ≥73%) though significant room for improvement in the other countries – with for example poor availability of protocols to support staff with differential diagnosis and in the case of Guinea-Bissau, no indicator scoring positively for >40% of the facilities assessed. Notably all facilities assessed in Mali used pre-operative ultrasound. As described in WHO (2019) ultrasound (as opposed to transillumination which is non-specific) and a good physical examination is the preferred diagnostic modality as it may rule in or out other scrotal pathologies, confirm Stage I-II hydroceles and identify hernias (17).

The results from domains 5 and 10 informed the training of surgeons, provision of Filaricele Anatomical Surgical Task Trainer (FASTT) training material ⁴ , and the development and implementation of guidelines to distinguish hydrocele from LF and non-LF (Mali), ensuring that the hydrocele surgery checklist is available in each facility.

The results on basic equipment and medicines highlight significant deficit with regard to the availability of the listed disposable and non-disposable items and medicines. However, as highlighted by Martindale et al. (7), such deficits can be resolved in the short term through purchasing additional supplies and in the longer term through strengthening stock management and supply chain processes. Related quality improvement actions taken (prior to the start of programme activities) as a consequence of these assessments included the provision of surgery boxes with instruments (scissors, forceps) and operating kits containing medicines and consumables to all assessed hospitals in the four countries and electrocautery machines being acquired for Côte d’Ivoire and Guinea-Bissau. In one facility in Guinea-Bissau, gas for resuscitation was recharged and technical staff were brought in from the capital (surgeons, instrumentalist, anaesthetist) in order to raise the hospital’s score beyond 75% and be able to carry out the training of surgeons and a hospital surgery camp.

Programme managers should carry out an in-depth analysis of these shortcomings and develop realistic action plans with the technical and financial support of partners and Ministries of Health to increase the provision of quality care.

The Hydrocele Surgery Facility Assessment Tool (HSFAT) and the DIP (WHO Direct Inspection Protocol) for lymphedema gave Sightsavers exceptional foundations to build from. While the digitisation of data collection tools has been previously demonstrated (7, 14) one of the main benefits of the system we report on in this paper are the automated dashboards and automated reports. These highlight challenge areas for each country and facilitate the creation of action plans, focusing on areas which need improvement. They also help programme managers easily complete the two indicators on the WHO EPIRF (Number of implementation units with at least one facility designated to provide recommended essential package of care and Total number of designated health facilities providing care) (11). Results can also be used to complete the elimination dossier by providing the number of health facilities evaluated as part of LF morbidity management. Future implementation of these assessments will include stronger training for LF focal persons in national programmes in these areas of data use.

There were some challenges faced while undertaking the assessments that should be taken into consideration while planning for future use of both tools. Insecurity in certain areas was a challenge, for example in Mali respondents from three health facilities who could not be reached on site because of insecurity were interviewed by telephone to collect all the information required for the assessment. Additionally, there was on occasion resistance to use the tool in the fear that results may not be flattering to the programme. This was mitigated through communicating the purpose of the assessments – which is to help identify challenges and improve services. Timing of the assessments was also sometimes a challenge, as Sightsavers requested countries complete them before the WHO’s two-year prior to dossier submission timeline. However, doing them early builds skills and familiarises health personnel so they are better prepared for the assessments needed for dossier development – when this perspective was shared, national programmes were agreeable.

In terms of data collection, the knowledge of survey teams/enumerators around lymphedema management was sub-optimum - pointing to the need for better training of survey teams in future assessments. In some cases elucidation of ‘intended’ responses may also be brought into question e.g. where positive scores for soap were linked to soap available for staff use (as opposed to lymphedema patients).

In terms of the analyses present in this manuscript, we acknowledge shortcomings related to the fact that the results from some of the non-scored indicators in the assessments (e.g. HFAT 10.15-10.23) have not been covered. An important finding was that the technique of excision of the tunica vaginalis as recommended by GAELF and WHO is not well known and applied in countries (17). The need for continuous training prior to the start of surgical activities in all countries was the first step in the process of implementation of hydrocele surgery. We also acknowledge that we have not presented LFAT results by facility type.

The HFAT and LFAT assessments have the potential of contributing to wider health system strengthening efforts and progress countries towards greater universal health coverage (UHC) - where all people, including people with disabilities, have access to the full range of quality health services they need, when and where they need them, without financial hardship. We note a few key areas where their use contributed more broadly to strengthening the health system. One is strengthening the capacities of personnel in digital data collection, data use, and in best practices in undertaking facilities assessments. Another is through strengthening the systems for acquisition of essential drug stock and equipment for health facilities – e.g. provision of drugs, consumables, surgical boxes, and cautery machines. LF programmes may wish to look to Liberia to learn from national experience in the integration of case management medicines and supplies into the national supply chain (18).

In Mali the low presence of basic washing kits in health facilities may in part be due to the fact that these items are considered to be non-medical by health systems and not part of the essential supply list, with an unwritten expectation that such materials would need to be furnished by individuals or external programmes. The Mali programme is planning advocacy efforts around integration of these non-medical items into the supply chain, especially in districts where lymphedema cases are recorded and in high numbers.

A lack in availability and readiness to provide high quality morbidity management care to patients for one condition is likely to be an issue for other health conditions as well. Sub-optimal scores on health facility assessments may indicate more systemic accessibility issues and point towards similar corrective actions. In the case of LFAT, these corrective actions may particularly benefit individuals requiring long-term care to manage the debilitating symptoms of other skin related conditions (e.g. inflammatory episodes of adenolymphangitis (ADL), onchocerciasis related skin disease, leprosy, etc.).

Results from more general domains (i.e. those not specific to LF), such as infrastructure-improve water source, have great significance for broader health service delivery and as such may be used to support the prioritisation of quality improvements more widely. Achieving this will require effective dissemination to decision makers such that this evidence can inform wider and more comprehensive national health policies, development plans and budget allocations. Collaboration, both within other departments in the MoH (e.g. national hospital evaluation agency) and beyond is pivotal.

Assessment results also highlighted the constant and ongoing need for health facilities to ensure disease specific guidelines for the treatment and monitoring of chronic disease in patients are kept updated and available to health workers. Moreover, these guidelines should be reinforced through ongoing refresher trainings of health staff.

In terms of longer terms planning in individual countries, Mali, for example have used the assessments as a catalyst: since these assessments were done, further LFAT and HFAT assessments have been supported in the Koulikoro region and the district of Bamako in Mali. At the time of writing, plans are underway to extend assessments to Kayes and Segou and mobilise funds for Mopti region. The country also plans to do advocacy with the WASH sector to improve water supply at the level of health facilities in the region, establish support groups for hydrocele and lymphedema patients in endemic communities and collaborate with nutrition support groups to improve the identification of lymphedema cases.

Based on feedback from the four countries who undertook assessments, Sightsavers will be updating the tools, including the addition of patient interview results into dashboards and reformatting of the final automated report. Sightsavers is also working on better aligning the content and scoring protocol of the LFAT with the WHO DIP.

This paper illustrates the value of using these assessment tools in countries at various stages towards the elimination of LF and with varying levels of LF morbidity management experience. The tools are available to all national LF programmes and implementing partners; interested parties can contact the corresponding author of this paper.