The polymeric materials Poly (D,L-lactide-co-glycolide) (PLGA) (lactide/glycolide ratio of 50:50), molecular weight 7 000–17 000 with acid end groups, carboxylic acid polyethylene glycol PLGA (carboxylPEGPLGA), Pluronic F127, 2-(N-morpholino)ethanesulfonic (MES) free acid, Tris (hydroxymethyl)aminomethane (TRIS),1-Ethyl-3-(3-dimethylaminopropyl)carbodiimide (EDC), N-hydroxysuccinimide free acid (NHS), Anti-PSMA antibody, hydroxylamine hydrochloride, sodium hydroxide pellets (NaOH), hydrochloric acid, fluorescein iso-thiocyanate (FITC), Hoescht-3342, 4′,6-Diamidino-2-Phenylindole (DAPI), tween80, and acetonitrile, were purchased from Sigma-Aldrich (St. Louis, MO, USA) and Merck (Pty) Ltd, Estate South, Modderfontein, Gauteng, SA. Docetaxel was obtained from DLD Scientific, Durban South Africa. For cell culture and experiments, Roswell Park Memorial Institute (RPMI) and Dulbeccos’s Minimum Essential (DMEM) culture medium, fetal bovine serum (FBS) and penicillin/streptomycin were purchased from Thermofisher Scientific, Waltham, MA, USA.

The preparation and characterisation of antibody-conjugated nanoparticles was described previously (Essa et al., 2025). Briefly, PLGA, or a mixture of PLGA and carboxylPEGPLGA, and docetaxel dissolved in acetonitrile served as the organic phase, while a 0.1% solution of Pluronic F127 in Millipore water served as the aqueous phase for fabrication of nanoparticles using the NanoAssemblr Benchtop (Precision Nanosystems Inc., Vancouver, BC, Canada. The resultant suspensions were stirred for 4 h at room temperature and pellets were obtained by three times centrifugation at 12,000 rpm for 30 min. PLGA-docetaxel nanoparticles, were the non-targeting nanosystem (NTN) while active targeting nanoparticles (ATN) were further functionalized by conjugation of the PLGA: PLGA-PEG-COOH with anti-PSMA antibody via carbodiimide chemistry. Fluorescent nanoparticles, called FITC-ATN were prepared in the same way, except FITC was used instead of docetaxel in the organic phase, and the preparation was conducted in the dark. The nanoparticle pellets were frozen at −80 °C and then freeze dried (Freezone 12 lyophilizer, Labcono, Kansas City, USA) for 24 h.

PC-3 and LNCap prostate carcinoma cells were obtained from Cellonex (Johannesburg, South Africa). Cells were confirmed by supplier to be free of mycoplasma. PC-3 cells were grown in RPMI culture medium, and LNCap cells were grown in DMEM/F12 culture medium (Gibco). The media were supplemented with 10% FBS (Gibco) and 1% penicillin/streptomycin (Gibco) as per cell culture protocols. The cells were cultured under sterile conditions in an incubator at 95% humidity and 5% CO2. For all assays, cells were counted using the LUNA II Automated Cell Counter (Logos Biosystems, Anyang-Si, South Korea) before seeding. A concentration of 10% DMSO was used as a positive control in cytotoxicity, apoptosis, cell cycle arrest, and spheroid growth inhibition assays. Cell culture medium was used as a negative control in these experiments.

PC-3 and LNCap cells were grown in RPMI and DMEM/F12 media, as prepared above and grown until 80% confluent. Thereafter, they were harvested and seeded in six well plates at a density of 2.5 × 104 cells/well for PC-3 and 5 × 104 cells/well for LNCap and grown until 80% confluence. Thereafter a scratch was made across each well with a 100 µL pipette tip, and cells were treated with whole media (control), and docetaxel, NTN and ATN. The minimum effective concentration of docetaxel (8 ng/mL) was determined previously (11) and confirmed during the assay timeline. Docetaxel was dissolved in DMSO and diluted to a final treatment concentration of 0.1% DMSO, which was also used as the vehicle control. Images were captured every 24 h over a 72-h period, and gap closure inhibition relative to untreated control was evaluated using ImageJ software, using Equations 1,2 . adapted from (14). The workflow of the assay is shown in Figure 1, adapted from (Grada et al., 2017). %   w o u n d   c l o s u r e = W t 0 − W t x W t 0 × 100 % (1)

Where Wt₀ is the initial wound area at time t0 and Wt_x is the wound area at time x, the time point of interest. %   c l o s u r e   i n h i b i t i o n = % W C c x − % W C t x (2)

Where WCc_x is the wound closure of the control group and WCt_x is the wound closure of the treated group at time x.

The MUSE™ Cell Analyzer (LuminexCorp, Austin, TX, USA) was used to detect miniaturized fluorescence emission from single cell suspensions in order to provide quantitative information on the effects of the nanoparticle treatments on apoptosis and the cell cycle of PC-3 and LNCap cells. A system check and complete clean cycle were run prior to experimentation. Additionally, the quick clean function was carried out in between experiments to prevent sample cross contamination and blockage of the capillary tubes.

Reactive oxygen species in cells were determined using the Fluorometric Intracellular ROS kit (Sigma-Aldrich, St. Louis, MO, USA). LNCap and PC-3 cells were seeded in 96 well plates at a cell density of 5 × 103 cells/well in DMEM/F12 and RPMI respectively. Twenty-four hours post cell attachment, cells were treated with 100 µL of the fluorometric master buffer assay mix and incubated for 1 h. Thereafter cells were treated with growth media (control) and the effective concentration of 8 ng/mL of docetaxel, NTN and ATN. Plates were read 72 h after treatment at excitation wavelength of 475 nm and emission wavelength of 520 nm. Fluorescence intensity was taken to be proportional to ROS activation as per assay protocol.

Spheroids were generated from the LNCap cell line using a method adapted from (Friedrich et al., 2009). Cell culture plates were coated with 1% agarose in PBS, and cell were seeded over this at different densities, with five replicates per density. Spheroids were monitored using bright field microscopy, and volumes were calculated using Equation 3 (16). V = a   b 2 2 (3)

Where V = volume of spheroid in µm³, a = major diameter in µm and b = minor diameter in µm.

Spheroids were measured on day 4 and treated with a range of concentrations of docetaxel, NTN and ATN. A treatment of 5% DMSO was used as a positive control for cell death. Seventy-two hours later (day 7), Alamar Blue assay reagent was added to the culture medium, and culture plates were incubated for 3 h at 37 °C. Thereafter, fluorescence readings were taken using 535 nm excitation and 595 nm wavelengths on the Viktor X3 multiplate reader (Perkin Elmer, MA, USA).

Spheroids were imaged and measured on day 4 and treated with media (control) docetaxel, NTN and ATN. Spheroids were measured 72 h later (day 7) and treated again. Thereafter, spheroids were treated and measured every 48 h (days 9, 11, 13, 15), and measured again at day 17. which was the endpoint of the assay. This schedule is adapted from the work by Friedrich and co-workers (Friedrich et al., 2009).

Experiments were conducted in the dark. Spheroids were seeded and treated with media (control) docetaxel, NTN and ATN. After 72 h, spheroids were washed three times with PBS and stained with 2 µm calcein and 4 µm ethidium homodimer using the LIVE/DEAD viability/cytotoxicity kit (Thermofisher, Waltham, MA, USA), according to the method by Petry and Salzig (17). The spheroids were counterstained using a 1:10,000 dilution of 1 mg/mL Hoescht 33,342 and incubated for 30 min at 37 °C. Thereafter, spheroids were imaged with the CELENA X Digital Imaging System (Logos Biosystems, Anyang-Si, South Korea) using the DAPI, EGFP and RFP fluorescent channels. Images were then processed using ImageJ sotware.

Experiments were conducted in the dark. Spheroids were seeded and treated with FITC-ATN for 24 h. They were then collected in microcentrifuge tubes, washed thrice with PBS, and fixed using 4% paraformaldehyde for 30 min. Following three further PBS washes, nuclei were stained with 1 μg/mL DAPI for 10 min. The spheroids were then mounted onto glass slides for viewing on the Zeiss LSM910 Lightfield 4D System with AiryScan 2 (Zeiss, Jena, Germany).

All experimental data were processed and analyzed by Origin software (version 8.5.0 SR1. OriginLab Corporation, U.S.). All results were represented by the mean ± standard error of three biological and three technical replicates, unless otherwise stated. Comparison of data between experimental groups was evaluated by one way analysis of variance (ANOVA) and Bonferroni adjustment, with p values less than 0.05 being designated as significant.

Here, images were captured at 0H (time of scratch) to measure the initial wound area, and thereafter the change in wound area was measured and calculated by Equations 1 and 2. Every 24 h for each treatment on each cell line. For the assay on PC-3 cells, all treatments had no significant difference in wound closure at 24 h and cells migrated to fill the wound area by 48 h (Figure 2). This indicated that none of the treatments had any noticeable or significant effect on wound closure.

As can be seen by Figures 3, 4A,C, the migration rate of both treated and control groups of LNCap cells was much slower than PC-3 cells, with the control group only reaching 61.0% ± 7.4% wound closure after 72 h. This is consistent with a recent study by Bossio and co-workers (Bossio et al., 2023) who achieved <75% wound closure of the control group of LNCap cells after 72 h. In this study, the NTN treated group showed significantly greater inhibition of closure than the docetaxel treated group at 48 (39.0% ± 1.1% for NTN and 24.6% ± 0.7% for docetaxel) and 72 h (48.1% ± 2.7% for NTN and 11.6% ± 7.1% for docetaxel), but displayed no significantly different inhibition percentage to docetaxel at 24 h, shown by Figure 4B. This could be due to the initial availability of the entire dose of docetaxel in the solution media, compared to the NTN system which showed a sustained release profile (11) and therefore caused more inhibition at the later time points.

The apoptosis profiles of treated groups are reflected in Supplementary Figure S1 for PC-3 cells and Supplementary Figure S2 for LNCap cells. The main apoptotic pathways measured were through early and late apoptosis, and there was no significant difference in the total apoptosis of native docetaxel (4.83% ± 0.4%), NTN (3.4% ± 0.80%) and ATN (4.25% ± 0.63%) in PC-3 cells. However, both nanosystems displayed lower necrosis profiles (top left quadrant in Supplementary Figure S1), with 2.27% ± 0.37% for NTN and 1.26% ± 0.07% for ATN than free docetaxel (4.12% ± 0.30%). There was no significant difference between the apoptotic profiles of both nanosystems on PC-3 cells.

Interestingly, there was also no significant difference in the apoptotic profiles and total apoptosis of both nanosystems on LNCap cells (8.32% ± 0.68% for NTN and 9.01 ± 0.90 for ATN), as shown by Figure 5, but both were significantly greater than the total apoptosis of free docetaxel, which was 3.82% ± 0.38%.

As suggested by Supplementary Figure S3 and reflected in Figure 6, there was no significant difference in the percentages of PC-3 cells in the G2/M phase across all treatment groups, indicating that none of the treatments could cause the G2/M phase mitotic arrest that is usually associated with docetaxel.

There was a significantly higher percentage of LNCap cell populations in the G2/M phase of both the groups treated with the nanosystems, compared to the untreated control and native docetaxel treated groups, as shown in Figure 6. The G2/M phase populations of the control and docetaxel treated groups which were not significantly different to each other, which implies that at this concentration, free docetaxel was unable to cause significant mitotic arrest. However, there were significantly higher percentages of G2/M phase populations in the NTN treated (36.4% ± 1.0%) and ATN treated (50.9% ± 1.2%) groups, as shown in Figure 6.

The ROS activation was analysed fluorometrically and the results showed no significant difference in the ROS activation levels between any of the treated groups relative to the control in PC-3 cells, as shown in Figure 7. However, the ROS activation levels of both nanosystems were significantly greater than the control and docetaxel in LNCap cells, while the level of the NTN treated group was not significantly different to the ATN treated group, also reflected in Figure 7.

In this study the seeding density of LNCap cells was optimized to generate uniform spheroids, using 10,000 to 650 cells per well (Supplementary Figure S4). Singular spheroids did not grow with the seeding density of 10,000 cells per well, while the spheroids that developed from the initial seeding density of 5000 cells/well displayed irregular morphology, as shown in Supplementary Figure S4. The formation of a necrotic core can be observed in some of the spheroids initially seeded with 5000 cells/well.

As Figure 8 shows, free docetaxel was not effective at reducing the viability of the spheroids at any of the tested concentrations, while the NTN treatment was significantly more effective than docetaxel alone at concentrations of 40 ng/mL and 50 ng/mL. However, the ATN treatment was significantly more effective than both treatments at 20 ng/mL and 30 ng/mL. For this study, 20 ng/mL was used as the effective concentration of free docetaxel and docetaxel in nano-systems for all further spheroid assays.

Treatments were conducted on days 4, 7, 9, 11 and 13. As shown by Figure 9, there was no observable difference between the morphology of untreated spheroids and the spheroids treated with docetaxel at all time points. Spheroids treated with the NTN showed cell death from day 11, while those treated with the ATN showed cell death from day 7.

Figure 10 shows that both untreated and spheroids grew exponentially, at a much faster rate than both the nano-system treated groups after day 9. This has been observed in previous inhibitory studies on spheroid growth (Brüningk et al., 2020; Kulesza et al., 2023). There was also no significant difference on spheroid growth between the untreated spheroids and docetaxel treatment at all time points. The endpoint of the assay was recommended to be at any point after 14 days, when the volume of the untreated spheroid reaches five times the volume before first treatment (Friedrich et al., 2009). In this study, this volume translated to 0.414 mm3 on day 15. Thereafter, one more measurement was taken at 17 days of spheroid growth.

As can be seen by Figure 10, there was significant inhibition of spheroid growth in the ATN treated group from day 11 (32.7% ± 4.5%) until day 17 (67.8% ± 5.5%) and in the NTN treated group from day 13 (21.0% ± 3.9%) until 17 (17.6% ± 4.6%).

Figure 11 shows the staining of live cells in the media, docetaxel, and NTN treated groups This is due to calcein, which is emitted by live cell esterase. The ATN treated group shows majority of the spheroid stained with ethidium homodimer, which corresponds to the disrupted cell membranes of dead cells. The nuclei are counterstained with Hoescht 33,342 in blue, and show a reduction in the size of the spheroids in the ATN group compared to the other treatment and control groups.

To determine if the efficacy of the ATN nanoparticles could be due to its penetration into the spheroids, we treated the group with fluorescent FITC-bound ATN nanoparticles. As Figure 12 shows, there is a distribution of the nanoparticles, shown in green, throughout the 3D structure.

Figure 13 shows the spheroid at different angles on the Y-axis, demonstrating that the nanoparticles are found throughout the spheroid surface and internal structure. This can also be viewed in the Supplementary Video S2, which shows the 3D rendering of the rotation of the spheroid along the Y-axis.

Cell migration is involved in many important cellular processes, including cell division, tumour cell invasion, angiogenesis and metastasis. The cytoskeleton of microtubules, which are tube-like intracellular structures that promote cell division, is implicated in cell migration. This cytoskeleton should be initially intact, and thereafter, the asymmetry of microtubules precedes directional migration (Wang et al., 2019).Taxanes such as docetaxel are potent microtubule stabilizers and have been found to influence migration by interacting with microtubules and interfering with their dynamics within the cell (Bahar and Yoon, 2021).

Cancer cell scratch wound healing assays can provide insight into the therapeutic efficacy of cancer treatments as the deregulation of cell migration can affect the above physiological processes associated with cancer cell phenotypes. The tracking of the inhibition of wound closure of cancer cells after being exposed to effective concentrations can indicate the metastatic potential of the treatment (Bahar and Yoon, 2021). The assay is used to evaluate cancer cell metastasis by the inhibition of “wound healing” caused by the treatments and the time required for healing (Bouchalova and Bouchal, 2022).

Wang and co-workers obtained similar closure results after 24 h of treatment with 1 nM (0.81 ng/mL) docetaxel (Wang et al., 2022). However, they also reported marked inhibition of wound closure on PC-3 cells, 24 h after being treated with 0.1 µM (80.8 ng/mL) of docetaxel (He et al., 2024). This signifies that docetaxel could have anti-migratory effects on PC-3 cells at higher concentrations, especially since the effective concentration in this study (8 ng/mL) was to the order of 10 lower than that used in He’s study and there was no effect of any of the treatments on migration, which could be due to the relatively low effective concentration of docetaxel used.

As shown by Figure 4D, the inhibition of wound closure by the ATN treated LNCap cells was significantly greater (p < 0.05, one-way ANOVA) than the other treated groups at all time points, with 49.8% ± 3.9%, 76.7% ± 6.8% and 89.4% ± 8.4% inhibition at 24. 48 and 72 h respectively. This can be attributed to the active targeting antibody moiety on the surface of the nanosystem that could have allowed for increased cellular uptake, facilitated by the PSMA receptors found on the surface of LNCap cells, leading (Wüstemann et al., 2019) to the intracellular inhibition of motility.

The earlier observation of no effect on wound closure of any of the treatments on PC-3 cells in Figures 2, 5A,B was attributed mainly to the PC-3 cell line being insensitive to the concentration of docetaxel used. The specific biological differences between the LNCap and PC-3 cell lines, such as androgen dependence and metastatic potential, are known to influence baseline migratory potential in cancer cell lines. For example, the work by Bose demonstrated that the upregulation of androgen receptors can modulate cell migration in cervical cancer (Bose et al., 2025), while Mehanna’s group recently described how metastatic potential dictates the rate of wound closure in breast cancer models, comparing MCF-7 and MDA-MB-231, which are cell lines with low and high metastatic potential respectively (Mehanna et al., 2025). In the context of prostate cancer, the PSMA-positive profile of LNCap and the PSMA-negative status of PC-3 are well-documented characteristics, confirmed in previous studies by Western blot (Ghosh et al., 2005), fluorescence-activated cell sorting (Feldmann et al., 2017), and immunohistochemistry (Runge et al., 2023). While the observed differences in migration and cell cycle arrest between the 2 cell lines in our study do reflect the intrinsic properties of the cell lines, the efficacy of the targeted versus non-targeted system within the LNCap line warrants further consideration. The targeted system achieved a significant 41% increase in migration inhibition over the non-targeted system in LNCap cells, whereas none was observed in the PC-3 model. It is highly probable that the presence of PSMA receptors on LNCap cells facilitates a more efficient cellular uptake of the system, potentially resulting in a higher intracellular release of docetaxel. However, given the biological complexity of these cell lines, the observed enhanced efficacy is likely a synergistic outcome of both the targeted delivery mechanism and the intrinsic sensitivity of LNCaP cells.

As shown by Figure 3, from 24 h after treatment, the wound spaces of the control and docetaxel treated LNCap cells are mainly occupied by cell projections from the edge of the wound, consistent with chemotaxis, which is an important mechanism in cell migration caused by functional chemical signals (SenGupta et al., 2021). However, from the 48-h time point, there is considerable cell shrinkage and detachment from the culture plate in the wound space of the NTN and more prominently in the ATN treated groups. This phenomenon is consistent with previous cancer cell migratory inhibition studies (Wang et al., 2019; Mishra et al., 2020), and is due to enhanced cellular uptake, toxicity and inhibition of intracellular physiological processes by the nanosystems.

Taxanes have been found to inhibit wound closure at concentrations lower than effective anti-proliferative concentrations (Wang et al., 2019; Lu et al., 2020), and this is consistent with the results shown in Figure 4, which still shows low inhibition of wound closure of free docetaxel on LNCap cells even though the effective concentration of native docetaxel used is much lower than reported anti-proliterative concentrations (Edmondson et al., 2016).

Cell death can proceed through different mechanisms, the most well-known being necrosis and apoptosis (Carneiro and El-Deiry, 2020; Liu and Jiao, 2020; Chaudhry et al., 2022). Therefore, the evaluation of apoptotic potential of treatments on cancer cells can provide information about their therapeutic efficacy (Lee and Lee, 2021). Yang and co-workers (Yang et al., 2019) reported low response levels when treating PC-3 cells with low concentrations of docetaxel and only achieved higher apoptotic levels (>10%) after treating with their measured inhibitory concentration of docetaxel. The results here, at the effective concentration of 8 ng/mL docetaxel correlates with the result of no observable inhibition of wound healing obtained with the scratch assay in Figures 2, 4A,B, suggesting that apoptosis could be a driving force in the inhibition of cell migration and motility in PC-3 cells.

The fact that there was greater apoptosis by the nanosystems compared to docetaxel in LNCap cells but not in PC-3 cells could be due to the greater sensitivity of the LNCap cell line to lower docetaxel concentrations, as described previously (Kim et al., 2021). The increase in total apoptosis of the nanosystems relative to docetaxel could be the effect of improved metabolic stability of the drug within the media and sustained intracellular release (Essa et al., 2025), ensuring a prolonged therapeutic concentration of docetaxel directly at the target site within the cells. This is distinct from any targeting ability, evidenced by the lack of difference in the total apoptosis between the targeted and non-targeted systems. These results are relatively consistent with several studies which show enhanced apoptotic mechanisms of nanosystems compared to free docetaxel (Chen et al., 2016; Rivero-Buceta et al., 2019; Bromma et al., 2022). In this study, the increase in apoptosis caused by both nanosystems on LNCap cells correlates with the change in cell morphology and detachment of the monolayer observed in the scratch assay and noted in Figure 3 on the NTN and ATN treated groups. This indicates that apoptosis could be a contributing factor to this phenomenon, but since the result is more pronounced in the ATN treated group, there could be factors other than apoptosis, driven by effects such as cell cycle arrest and inhibition of proteins linked to metastasis (Vargas-Accarino et al., 2021) at work here.

The anti-cancer mechanism of docetaxel is well characterised (Imran et al., 2020; Ehsan et al., 2022; Hashemi et al., 2023). In this study, the cell cycle profiles of both cell lines were evaluated with all treatments to compare the mechanism of action of docetaxel and the nanosystems, and their corresponding effects on the cell cycle. Lv and co-workers reported a modest increase from 20% to 30% in the G2/M population of PC-3 cells treated with 20 nM (16.1 ng/mL) docetaxel (Lv et al., 2017), but it was not stated if this difference was significant, while Cancino’s group (Cancino-Marentes et al., 2021) and Meidavilla-Varela’s group (Mediavilla-Varela et al., 2009) both reported no significant differences when treating PC-3 cells with 25 nM (20.2 ng/mL) and 3 µM (2 424 ng/mL) docetaxel respectively. In this work, the fact that mitotic arrest did not occur correlates with the results obtained for the wound healing assay where there was no inhibition of cell migration and wound closure across all treatments in the PC-3 cell line.

Lima’s group reported a 36% increase (Lima et al., 2021) and Lv’s group reported a 47% increase (Lv et al., 2017) in the LNCap G2/M phase populations, both groups having treated with 20 nM (16.1 ng/mL) docetaxel. This could suggest that for the LNCap cells in this study, the concentration of free docetaxel used (8 ng/mL) was too low to induce cell cycle arrest in the G2/M phase. Similar results were obtained by Cao and co-workers, who prepared non-targeted and targeted PLGA nanoparticles loaded with docetaxel, and reported 30% and 60% of G2/M phase populations of LNCap cells treated with the targeted and non-targeted nanosystems respectively (Cao et al., 2016). In this study, the increase in G2/M phase populations indicate the proportional induction of mitotic arrest in the NTN and ATN treated groups. The populations translated to 15.8% ± 2.8% and 30.3% ± 3.0% arrest relative to the control for NTN and ATN treated cells, respectively. The difference in efficacy can be attributed in part to the targeting ability of the ATN, allowing for increased cellular uptake via the interaction between the targeting antibody and the PSMA receptors on the surface of the LNCap cell line, whereas the NTN does not have anti-PSMA ligands on its surface and therefore is not able to take advantage of the PSMA receptors on the LNCap cell surface. This could explain the proportional increase (ATN > NTN > DOC) in the inhibition of cell migration observed earlier, as mitotic arrest caused by prevention of microtubule polymerisation leads to inhibition of cell motility (Nakonechnaya and Shewchuk, 2015). A contributing factor is that the cell cycle arrest would have disrupted the G2/M checkpoint that prevents cells with damaged DNA from undergoing mitosis (Ventura and Giordano, 2019). Additionally, the G2 checkpoint arrest could have inhibited the formation of maturation/mitosis promotion factor (MPF), which promotes cells from the G2 to the M phase in the cell cycle (Zhang et al., 2021), thereby preventing cell migration.

ROS is a normal by-product of healthy physiological processes (Katerji et al., 2019). However, overproduction of ROS can trigger apoptosis and therefore the measurement of the production of ROS induced (ROS activation) caused by proposed cancer treatments can be a valuable indicator of their apoptotic potential (Zaidieh et al., 2019). In this study, we observed superior ROS activation of both nanosystems compared to docetaxel but there was no observable increase provided by the targeted versus the non-targeted system. The results of the ROS activation by both nanosystems correlate with the apoptosis results which show no significant difference between the nanosystems but more apoptosis of both nanosystems compared to docetaxel.

3D multicellular spheroids have been shown to mimic the characteristics of in vivo tumours more accurately than 2D models and there has been widespread discussion of the clinical relevance of the use of in vitro 3D models in studying tumour biology (Filipiak-Duliban et al., 2022). Additionally, the 3D structure of cancer cell spheroids in assays to test efficacy of treatments are more indicative of their projected performance in vivo, as the treatment is only able to penetrate outer layers of the spheroids, much like in in vivo tumours and unlike 2D cell culture where the treatments are available in the growth medium, to a higher percentage of the cells attached to the culture plate (Perche and Torchilin, 2012). The necrotic zone indicates the inner areas of the spheroid which do not have access to oxygen and nutrients as the outer layers do, and therefore form a zone of toxic waste products (Dini et al., 2016).

The selection of effective docetaxel concentrations for 2D cells (8 ng/mL) was based on the minimum effective concentration established in our previous optimization studies (Essa et al., 2025). This was defined as the minimum dose required for a biological response of a 50% reduction of cell viability in PSMA-positive LnCap cells. B selecting this dose for both LNCap and PC-3 lines, we aimed to isolate the efficacy provided by the targeted delivery vehicle from the baseline cytotoxicity of the free drug. The selection criterion of the effective concentration of 20 ng/mL for the spheroid model was the same, based on the viability results in Figure 8. The free docetaxel treatment was inactive in PC-3 cells, which is likely due to the resistance of this androgen-independent cell line to low-dose taxane therapy (Liu et al., 2013).

Notably, the minimum effective concentration of the ATN treatment for spheroids is greater than that for the 2D cells (20 ng/mL for spheroids compared to 8 ng/mL for 2D LnCap cells). This could be due to the inadequate penetration of the treatments through the barrier of the extracellular matrix and dense interstitial fluid characteristic of the 3D spheroid structure (Brüningk et al., 2020), and also explains the lack of efficacy of free docetaxel on the spheroids shown in Figure 8 (80%–90% viability at all treatment concentrations) compared to LnCap 2D cells (50% viability at 28 ng/mL of free docetaxel), (Essa et al., 2025). This is consistent with the research of Xu’s group that required twice the concentration of docetaxel treatments to cause spheroid viability from 80% to 60% (Xu et al., 2022). The fact that the targeted system achieved significant cell death, migration inhibition and G2/M arrest at the concentrations that free docetaxel was ineffective highlights the therapeutic gap that the targeted system was designed for. The enhanced efficacy of the targeted system on the spheroids implies that it potentially improves the solubility and penetration of within the complex 3D microenvironment.

These results correlate with the relative reduction in viability between the treatments, shown in Figure 9, and demonstrates the ability of the targeted system to penetrate the 3D spheroids more efficiently than either the NTN or docetaxel. It also corresponds with the viability measurements in Figure 8, which shows no significant difference between the 20 ng/mL docetaxel treatment and the control. Enhanced targeting efficiency was reflected in the earlier and more robust growth inhibition observed with the ATN system. Moreover, both nano-systems demonstrated higher efficacy in spheroid growth suppression compared to free docetaxel starting at day 13. Additionally, Figure 9 shows morphology changes and necrosis, by the darkening of the ATN and NTN treated spheroids from days 7 and 11 respectively. This is earlier than the inhibition observations (days 11 and 13 respectively), indicating toxicity at the cellular level that occurs before changes to spheroid size.

As with in vivo studies, the relevant dosage of treatments should be optimized for 3D culture models (Friedrich et al., 2009). In previous studies (Bäcker et al., 2016; Eder et al., 2016; Jouberton et al., 2022; Raitanen et al., 2023), cell viability studies were conducted on spheroids with ranges of concentrations in order to determine the effective inhibitory concentrations of treatments for 3D multicellular tumour spheroids inhibitory concentrations (Vinci et al., 2012). Much like 2D culture measurements, the results of cell viability assays vary greatly between studies, and there have been reports of inhibitory concentrations of docetaxel on LNCap spheroids between 70 nM (56.6 ng/mL) (Edmondson et al., 2016) and 25 µM (Jouberton et al., 2022). These concentrations are much higher than what was used in this study (20 ng/mL), and it could be possible that this concentration was too low for docetaxel alone to have an effect on spheroid growth and health.

Karendish and co-workers reported on docetaxel loaded polymersomes targeted with folic acid and showed a decrease in cell viability of 88%–55% between their non-targeted and targeted systems on LNCap spheroids (Karandish et al., 2016). They tested a lower effective docetaxel concentration of 10 nM (8.1 ng/mL), while in this study there was no significant difference between the NTN and ATN treatments at 10 ng/mL. However, at an effective concentration of 20 ng/mL, the cell viability of the spheroids was reduced from 69.5% ± 6.0% to 43.4% ± 6.4%, indicating a similar improvement of the targeted system.

In order to explore the mechanisms of cell death within the spheroid structure, the effect on intracellular processes from intact spheroids should be determined. However, a limitation of this study was the analysis of these mechanisms in the 3D format. We encountered difficulty preparing viable single cell suspensions from the whole spheroids and the 2D cell migratory assay was not compatible with the 3D format. Therefore, we investigated the induction of these mechanisms by targeted nanoparticles in the 2D format and normalised the results against the activity on PC-3 cells, which does not express PSMA receptors, as well as with NTN, the non-targeted nanoparticles. A qualitative analysis of cell migration and invasion in spheroids was described by Massaro’s group, who observed a decrease of cellular expansion in the outer proliferative layer of melanoma spheroids (Massaro et al., 2017). In our study, the outer layer of the treated LNCap spheroids showed increased cell death, contributing to the inhibition of spheroid expansion in Figure 9. The study by Massaro also described cell cycle arrest in the G2/M phase of 2D SK-Mel melanoma cell lines which is consistent with our result of nanoparticle induced G2/M cell cycle arrest in 2D LNCap cells. In our work, the difference between the targeted and non-targeted systems (observed in the PSMA-positive LNCap cells) on G2/M phase arrest and 3D spheroid growth inhibition points to a targeting-effect. In contrast, the increases in apoptosis and ROS generation, caused by both nano-systems, appear to be more characteristic of a general nanoparticle-formulation effect.

The biological significance of the targeted nanoparticle treatment on 3D spheroids is evidenced here by the reduction in cell viability and the nanoparticle penetration through the spheroid. The enhanced efficacy of the ATN system correlate with the reduction in cell viability (but no significant reduction of viability in the NTN or free docetaxel treatments) observed earlier. This observation is also consistent with the work of Karendish’s group, who observed only 55% spheroid viability in their targeted treatment group, and >86% cell viability in other treatment groups (Karandish et al., 2016).

The presence of fluorescently tagged ATN’s in the spheroids implies that the nanoparticles would have had access to the binding sites in order to enter the cells, and release docetaxel, therefore causing enhanced cell death in this treatment group. This efficient distribution of nanoparticles throughout the 3D structure of the spheroid could be due to the interaction of the targeting ligand on the nanoparticle surface with the PSMA receptors on the surface of the cells in the spheroid. The penetration of the nanoparticles could also be enhanced by the presence of the PEG groups on the PLGA core, which has been reported to reduce electrostatic interactions with the extracellular matrix (Tomasetti et al., 2016) (62), and thereby could increase the diffusion of the nanoparticles through the spheroid. This effect has also been reported by Tchoryk and co-workers who observed efficient penetration of pegylated poly (glycerol adipate) nanoparticles into the core of HCT116 colorectal cancer spheroids (Tchoryk et al., 2019).