Aquaculture is expected to supply two-thirds of the world’s fish by the year 2030, underscoring the need for continuous improvement (FAO, 2022). Feeding plays a pivotal role in determining the success of aquaculture, with significant influence on various aspects such as operational costs, fish growth rates, overall health and welfare of the aquatic organisms, and the environmental footprint of the industry. It’s important to note that feeding requirements in aquaculture are far from static; they can be notably impacted by factors like the fish size, the environmental conditions, and the health status of the reared group (Sun et al., 2016; Zhang et al., 2020). Thus, innovative methods to optimize feeding practices become imperative in achieving the triple bottom line of economic viability, social responsibility, and environmental sustainability within the realm of aquaculture.

Efficient feeding can be achieved by monitoring both behavioral and water quality parameters. On one hand, water quality parameters, such as temperature, oxygen and pH, can impact the metabolism of the fish and consequently their appetite (Kramer, 1987; Zhou et al., 2018b; Volkoff and Rønnestad, 2020; Assan et al., 2021). On the other hand, changes in metabolism are associated with changes in activity. The variations in activity levels can therefore serve as indicators of appetite, revealing the feeding needs (Chen et al., 2023; Conde-Sieira et al., 2018; Zhou et al., 2018b, Zhou et al., 2019). Studying activity can, thus, enable efficient feed utilization, preventing overfeeding and associated environmental implications, while avoiding stress and health issues from underfeeding. In addition, tailoring feeding practices based on fish behavior can lead to lower production costs and promote well-being. A comprehensive understanding of fish behavior can therefore empower responsible feeding practices, benefiting fish welfare and the environment, making it vital for sustainable aquaculture practices.

Fish display distinct and diverse behavioral responses to external stimuli and internal requirements. Numerous research studies have delved into these behavioral adaptations when it comes to feeding. Hansen et al. (2015b) demonstrated that differences in the internal states of fish within shoals influenced their individual spatial positions, with hungrier individuals tending to position themselves at the forefront of the group. Hunger also impacts activity patterns, as insufficient food prompts animals to engage in increased locomotor behavior associated with searching and exploration (Miyazaki et al., 2000; Priyadarshana et al., 2006; Hansen et al., 2015a) and greater overall swimming and turning speeds compared to well-fed fish (Hansen et al., 2015a; Killen et al., 2017). It can also increase the aggression between individual fish for food (Huntingford et al., 2012a, Huntingford et al., 2012b). Satiation levels are, therefore, tightly linked to specific behavioral expressions in aquatic environments.

Long efforts have been made to identify indicators, including behavioral, that can detect satiation levels and regulate feeding for different species. Initial studies attempt this using visual observations or indirect behavioral metrics. Brännäs and Alanärä (1993) for example, have recorded the biting activity of the arctic charr (Salvelinus alpinus) in tanks, while Fast et al. (1997) the feed consumption using cell loads to understand the satiation level of Chinese catfish (Clarias fuscus). Others have applied hydro-acoustic sensors for detecting feed pellets in sea cages for feeding control (Juell, 1991; Juell and Westerberg, 1993; Juell et al., 1993). Fang and Chang (1999) used a reflective type of photoelectric sensor to detect the gathering behavior of the fish. In other studies, an infrared photoelectric sensor was used to observe the collective behavior of Cyprinus carpio (Zhao et al., 2017), and developed an intelligent feeding control system for eel breeding (Chang et al., 2005). Parra et al. (2018) designed and deployed Low-Cost Sensors for monitoring the water quality and fish behavior in aquaculture tanks during the feeding process.

Acoustic telemetry systems have also been adopted by the aquaculture industry and have been proven valuable for understanding biological processes (Hussey et al., 2015; Abecasis et al., 2018; Føre et al., 2018; Schwinghamer et al., 2019; Villegas-Ríos et al., 2020; Svendsen et al., 2021). These systems could enable feeding control as they can wirelessly transmit real-time data, enabling fish farmers to promptly respond to changes in group behavior, such as during feeding (Thorstad et al., 2013; Føre et al., 2018) However, their invasive deployment may influence fish attributes, potentially leading to erroneous data (Hussey et al., 2015; McKenna et al., 2021; Georgopoulou et al., 2022; Runde et al., 2022). Moreover, the expense and limited deployment (An et al., 2021) to a few individuals result in a small, potentially unrepresentative sample size for farm-wide monitoring, restricting current use to feeding control.

Demand or self-feeders have also been developed to enable fish to feed according to their appetite (Alanärä et al., 2001; Noble et al., 2007a, Noble et al., 2007b, 2008). Self-feeders activate when fish trigger them, releasing food from a dispenser (Alanärä et al., 2001). These feeders aim to enhance feeding in aquaculture by aligning feed delivery with fish appetite, leading to improved growth and reduced feed wastage (Talbot et al., 1999; Noble et al., 2007b). Additionally, their implementation could potentially decrease competition, thereby enhancing fish welfare (Noble et al., 2008). Despite the positive performance outcomes observed in several cases (de Mattos et al., 2022), the use of self-feeders comes with challenges. Their operation depends on specific individuals and social hierarchies, which may result in growth inhomogeneity within a stock (Arechavala-Lopez et al., 2020. Moreover, deploying self-feeders in open cages poses challenges, especially in larger cages with a high number of individuals. Fish in marine cage environments often exhibit variable appetites due to factors such as changes in water temperature, environmental conditions, and the health status of the fish. While social learning and training theoretically hold the potential to address these inhomogeneities, there has been no successful demonstration of social learning as an effective tool for aquaculture to date (Macaulay et al., 2021). While many small-scale experimental studies report promising results, few have successfully translated to commercial scale, highlighting a gap between theoretical concepts and practical application. Caution is warranted against extrapolating results from small-scale studies to commercial situations.

In the past decade, there has been a recent surge in scientists proposing methods to quantify fish behavior using computer vision techniques (Xu et al., 2006; Niu et al., 2018; Zhou et al., 2018a; Georgopoulou et al., 2021) and only recently, from 2016, they have applied more elaborated methodologies, such deep learning (Macaulay et al., 2021). These techniques offer an automated, non-invasive, and cost-effective means to record behavioral parameters. To describe these behaviors, images captured are converted to statistical results using computer vision models, and then some behavioral parameters are estimated. Categorically, these methods can be distinguished based on several criteria: the species under study, the adopted methodology (ranging from simple computer vision techniques to more sophisticated machine learning models such as convolutional neural networks-CNN), the applicable system (e.g., recirculating aquaculture systems-RAS, cages), the specific research questions they aim to address [e.g. feed behavior, abnormal behavior such as aggressive behavior, disease states, environmental factors, stress, and daily swimming behavioral variations (Yang et al., 2021)], and the type of behavioral parameters studied. These parameters can be direct, such as the swimming speed, direction, spatial distribution, and turning angle (An et al., 2021), or indirect, such as the changes in the water surface due to higher activity and the loss of feed pellet due to low activity. Here, we focus on the methodologies that specifically have been applied for feeding monitoring and control. An et al. (2021), Li et al. (2020), Niu et al. (2018), Wang et al. (2021), Yang et al. (2021) and Zhou et al. (2018b) offer a thorough investigation and discussion on computer vision techniques used for monitoring fish behavior in other contexts.

In the realm of applicable systems, a concentrated effort has been directed toward recirculating aquaculture systems (RAS) but usually the studies involve a limited number of individuals (Foster et al., 1995; Parsonage and Petrell, 2003; Xu et al., 2003, 2006; Papadakis et al., 2012; Lee et al., 2013; Liu et al., 2014; Papadakis et al., 2014; Al-Jubouri et al., 2017; Zhou et al., 2018b, Zhou et al., 2018a; Han et al., 2020; Wang et al., 2021; Kong et al., 2022). Some of these investigations have developed methodologies centered on indirect metrics, such as counting uneaten pellets using simple computer vision techniques (Foster et al., 1995; Parsonage and Petrell, 2003). More recently, others have proposed a feeding control method in RAS, which involves integrating pellet count using more elaborated methodologies, i.e., a feed pellet counting module using convolutional neural networks (CNNs) (Wang et al., 2022).

Turning to methodologies directly monitoring the fish behavior (Papadakis et al., 2012; Lee et al., 2013; Papadakis et al., 2014; Zhao et al., 2017; Al-Jubouri et al., 2017; Zhou et al., 2018b, Zhou et al., 2018a; Han et al., 2020; Kong et al., 2022), Lee et al. (2013) introduced the Sustainable Aquaculture Feed System (SAFS), utilizing simple computer vision techniques to detect fish presence around the feeding area and evaluate their appetite. Another approach by Liu et al. (2014) involved defining an index for measuring the feeding activity of Atlantic salmon shoals in an RAS through the subtraction of two consecutive frames. Alzubi et al. (2016) employed a combination of traditional computer vision and more elaborated support vector machine to analyze fish activity and determine if fish were actively fed. Zhao et al. (2017) proposed a method characterizing fish groups’ appetite based on their dispersion, interaction force, and the degree of change in the water flow field. More recently, there has been a growing body of work developing methodologies based on CNNs to classify feeding-related activity into different classes (Han et al., 2020; Kong et al., 2022). Wang et al. (2022) used a multi-task CNN that contained also a feeding activity classification module, that could binary classify fish group into active or inactive.

In contrast, fewer studies have addressed systems applicable to sea cages, with a tendency towards indirect measures of fish appetite, such as detecting feed loss using cameras (Hu et al., 2021) or monitoring surface activity during feeding (Ubina et al., 2021; Hu et al., 2022; Pradana and Horio, 2022). Maløy et al. (2019) considered both temporal and spatial flow by combining a three-dimensional CNN (3D-CNN) and an RNN to form a new dual deep neural network. The 3D-CNN and RNN were used to capture spatial and temporal sequence information, respectively, thereby achieving recognition of both feeding and nonfeeding behaviors in salmon.

In general, previous work focused on developing feeding monitoring tools in RAS, with a limited number of fish and more controlled environments. In addition, they controlled a small number of parameters and did the analysis for short-time intervals. As a result, there is a significant lack of long-term experiments specifically designed to be applied in industrial aquaculture settings, particularly in cages. In these conditions, the feeding frequency and quantity of feed are typically determined based on feeding tables and farmers’ experience, with limited adaptability to actual fish feeding needs (Benhaïm et al., 2012; Wang et al., 2017). Furthermore, the use of automatic feeders without any control may result in cases of both, over-feeding and underfeeding (Ang, 1998; Petrell and Pee Ang, 2001; Liu et al., 2014).

Considering the above, we have developed a system capable of continuous, real-time monitoring of European seabass feeding behavior in sea cages using single vision underwater cameras. The system allows for the automated detection and tracking of individual fish, as well as the identification of their feeding actions and progression. Within this framework, we have introduced two parameters for control purposes: the speed and the feeding behavior index (FBI). While other parameters, such as polarization and fish turns, were also considered we focus on assessing how these two behavior-based indicators, speed and FBI, respond across various feeding scenarios including feeding frequency, feeding quantity, and feeding duration. Building on previous research, we anticipate two key findings: Firstly, the fish are likely to exhibit increased activity leading up to feeding events, suggesting an anticipatory response. Secondly, as the fish approach satiation, a decline in their activity levels is expected.

The current work is organized in the following sections: first, we describe the methodology we used to develop the continuous real-time tracking system and to extract the behavioral parameter. Then, we discuss the experimental design of the different feeding scenarios. Next, we present the results of the fish behavior parameters for the different feeding scenarios. Last, we discuss possible methodologies that could help us achieve feeding control.

In the present study, our objective was to address the lack of feeding control systems in sea cages. For this, we developed a system capable of continuously monitoring the feeding behavior of European seabass within the cages. To achieve this, we employed a combination of AI models (YOLO and DEEPSORT) along with computer vision techniques. To test our system and the sensitivity of the parameters to feeding we conducted a long-term experiment with variable feeding scenarios. We examined two different behavioral parameters, the group speed, and the feeding behavioral index. All tested parameters varied for different feeding scenarios suggesting they could be used for feeding control. Our findings suggest that fish exhibit distinct behavioral patterns in response to various feeding situations, and for both, the speed and feeding behavior index, can identify threshold values that correspond to satiation levels, facilitating controlled feeding. To our knowledge, this is the first study to have developed a continuous, real-time monitoring system for feeding control in sea cages and discusses reference curves that can facilitate feeding control.

Our monitoring system exhibits both advantages and limitations. On one hand, we successfully developed an online system capable of individual-level fish tracking. However, we observed that the tracking performance decreases with increasing shoal density, particularly during feeding. To address this, further annotation is required to improve tracking in such scenarios. To monitor behavior in high densities, we developed a group-level behavioral metric, the feeding behavior index (FBI). Although the FBI remains unaffected by fish density, it is negatively impacted by varying lighting conditions. While we partially addressed this issue by removing the background, additional refinement is necessary. We also tried our individual-level tracking system for different species, i.e., the gilthead seabream and the salmon, and without extra annotation, we could successfully track them in different farms (unpublished data). This is because our camera points upwards and the annotation includes fish that have very high contrast with the background, with a dark silhouette. Therefore, fish from different species look similar enabling flexibility for the selection of species.

Speed showed significant sensitivity, both quantitatively and qualitatively to the different feeding scenarios. Our findings show, in accordance with previous works (Miyazaki et al., 2000; Priyadarshana et al., 2006; Hansen et al., 2015a, Hansen et al., 2015b), that fish increase their speed in response to higher hunger levels. More specifically, there was higher speed in the morning in comparison with noon and afternoon, higher speed if the fish are fed once in comparison with multiple feedings, and higher speed in reduced feeding in comparison with normal feeding. Other studies, i.e., of Riddell and Webster (2017), in a different setup offer contrasting findings, as they show that hunger is not necessarily correlated with the time that individuals find and approach prey patches. This deviation could be explained by the differences in the species studied, their social structure and the small number of fish in the group (Hansen et al., 2020) and additionally by the fact that in our case the fish group has been habituated to receive feed in specific times at a specific area of their environment. During no feeding we did not see a significantly increased speed, during the whole day outside the feeding times something that we would potentially expect. One explanation for this could be that the prolonged hunger acts as a chronic stressor and the group adapts to conserve energy (Lankford et al., 2005). Although we see a difference in the speed between reduced feeding and overfeeding, this was not significant. This could be either because the level of hunger for individuals of the body size used in the trial was not perceived as chronic stress, or due to the duration of each trial that did not last for more than 10 days, a period that probably is not sufficient for the applied restriction on feed to be considered as a chronic stressor for E. seabass for which longer periods are needed (Santos et al., 2010). Further experiments are required with longer trials of different feeding quantities.

We detected significant differences in the speed asymmetry between different times only when the frequency of feeding was three times daily and not in other feeding frequencies. This could show that the time interval between consecutive feeding events can affect the response of the fish on feeding. Fish that feed more frequently have on average lower speed and probably weaker anticipation for food. Weaker anticipation would be seen as a more symmetrical behavior around feeding times. As the normal feeding schedule for this fish stock was to feed once per day, the fish showed a more symmetrical behavior in the morning. The fact that we did not detect respective significant differences when the frequency was two meals per day could suggest that the effect of the time interval between feeding events on the fish behavior could also be important and needs to be tested. Depending on the digestion rate (which also depends on temperature, feed composition and oxygen availability), stomach may not empty sufficiently for the fish to feel hungry again, until several hours have passed from their last meal (Adamidou et al., 2009; Gilannejad et al., 2019). The hypothalamus is responsible for controlling appetite and integrates information from different sources, i.e., the endocrine signals, the nutrient levels and the presence/absence of food in the gastrointestinal gut (Rønnestad et al., 2017). The differences seen could also be explained by the circadian rhythms and the environmental variability (Assan et al., 2021), something that should be tested in the future. Previous studies on different species demonstrated that the feeding frequency can be optimal under specific scenarios there must be an optimal feeding frequency (Paspatis et al., 2000; Hassan et al., 2021; Priya et al., 2021) in which the fish do show minimal stress levels, optimal energy expenditure and higher satiation levels.

The frequency of feeding has a significant impact on the fish’s behavioral response. While it is a common practice to feed fish once daily, this approach may lead to heightened excitation levels during feeding time, potentially indicating increased energy expenditure, elevated stress levels, or more instances of abrasion, negatively affecting the fish’s welfare (Rønnestad et al., 2017; Assan et al., 2021). This could suggest that providing multiple meals in a day would be better for the fish. However, it’s important to note that these observations are speculative, as it remains challenging to directly link the higher speed to the fish’s actual satiation level (Adamidou et al., 2009; Gilannejad et al., 2019). Further research could be therefore suggested to sample fish stomachs and/or blood for satiety markers. Our hypothesis arises from comparing the speed between different feeding frequencies and feeding times, but as well between normal and reduced feeding quantities, where we observed a substantial increase in activity when fish are underfed. Nevertheless, we must carefully define welfare in fish and identify reference behaviors, deviations from which could serve as indicators of hunger. Further studies are required for this.

The feeding behavior index (FBI) also showed sensitivity to different feeding scenarios. The average FBI showed a minimal plateau at noon and was high during morning and afternoon. The asymmetry of the excitation increased in multiple feedings, something that could be explained as in the case of speed asymmetry. From our analysis we observe that the excitation lasts longer when fish are underfed and thus, feel hungry. However, the opposite happens when fish are fasting. During this period, the fish show excitation for shorter times, suggesting that during long-term fasting fish adapt their behavior and the energy expenditure (Gilannejad et al., 2019). To our knowledge, there are no similar studies that use a similar index.

We have not identified any existing work that introduces the Fish Behavioral Index (FBI) in cages in the manner presented here. While there are studies that quantify population activity and its progression by detecting changes in the water surface (Atoum et al., 2015), none directly quantifies changes in fish density within cages. Another set of studies (Zhou et al., 2018b) introduces a similar index, the Feeding Behavior Quantification Factor (FIFFB), reflecting fish aggregation. However, these studies employ Delaunay triangulation in a more controlled and static environment, specifically in Recirculating Aquaculture Systems (RAS). In contrast, our presented metric, the FBI, is distinct and designed for dynamic environments such as cages. Further research (Zhao et al., 2016) has quantified shoal dispersion using a statistical joint histogram related to motion vectors derived from optical flow analysis. However, this metric differs from the FBI, and its applicability is limited to static environments, specifically RAS systems. In summary, our approach with the FBI in cages is novel and distinct from existing methods, offering a valuable contribution to understanding fish behavior in dynamic aquatic settings.

The two variables tested here, the speed and the FBI, showed some differences in their sensitivity to the variable feeding scenarios. The FBI showed a minimal plateau at noon while it was high during morning and afternoon. The speed variable is dynamic and although here it is studied at the group-level, is directly affected by the metabolic state of the fish and the fish energy requirements and reservoirs. In contrast, the FBI is a spatial variable that shows the crowding behavior of the group, and although dynamic, it is not directly linked to the metabolic state of the fish. The fact that the FBI showed a minimal plateau during noon, and higher activity during morning and afternoon suggests that this is an indicator that expresses the specific dual (diurnal/nocturnal) feeding behavior of seabass, i.e., during morning and dusk mostly than during the day.

We also showed that the activity expressed by the FBI can be grouped into four different clusters of behaviors, one pre-feeding cluster, two feeding clusters (the first that shows the feeding behavior at the initial period of the feeding and the second the behavior during the relaxation of the feeding) and the post feeding cluster. In addition, we showed that the duration of these feeding clusters could be affected by the different feeding quantities. This methodology could therefore be used in the future to achieve automatic detection of the satiation state of the fish and could help to control feeding better. Other potential tools that could help us predict and control feeding would be the detection of the transition point between the feeding and post feeding clusters. We have preliminary unpublished results, that show that the use of changepoint analysis and the prediction of the future value of FBI using neural networks could be very useful for the early detection and adjustment of feeding needs of the fish. Despite not yet achieving real-time feeding control, our work encompasses the development and presentation of valuable tools for inferring fish satiation levels. Consequently, there are two feasible approaches to controlling feeding: asynchronous and synchronous. The first involves evaluating today’s feeding and adjusting the quantity for the following day accordingly. Alternatively, we can anticipate the evolution of the control parameter using neural networks and make feeding adjustments when the prediction indicates that a specific threshold has been reached.

Until now, we showed that our system can reveal important information on the feeding state at the group-level. As shown previously, disparities in the internal condition of fish can serve as a significant source of heterogeneity within animal groups (Wilson et al., 2019; Jolles et al., 2020). It would be, therefore, very interesting to see if this heterogeneity of the internal satiation state of the fish is also expressed at the group-level. If this is the case, then, by designing appropriate experiments, we could work to identify specific behavioral parameters that can quantify the percentage of the hungry fish in the cage. Previous laboratory studies have already shown that differences in the hungry/satiated ratio can impact also the collective behavior of the fish (Wilson et al., 2019). However, these studies are controlled and with a limited number of individuals. Findings such the above but at large scales in farms, would be very valuable for the optimization of the feeding and the improvement of the welfare of the fish.

Changes in the main parameters of the farm environment (e.g., water temperature, dissolved oxygen concentration, pH value, and ammonia nitrogen compounds) can directly affect the appetite and food intake of fish (Stoner et al., 2006; Remen et al., 2016). For example, when fish eat, the concentration of local dissolved oxygen may decrease. Uneaten pellets deposit on the bottom of the water also causes changes in dissolved oxygen and ammonia nitrogen compound concentrations (Skøien et al., 2016; Li et al., 2017). Although the influence of these parameters on the feeding behavior was not part of the current study and are highly influenced by the hydrodynamics in the farm, especially in cage aquaculture, we believe it is necessary to study these in the future. Previous studies have shown that fish have different energy demands at different temperatures (Krumschnabel and Wieser, 1994; Krumschnabel et al., 1997; Volkoff and Rønnestad, 2020). Therefore, differences in the environmental conditions could potentially demand adaptation of the reference curves and the satiation thresholds.

To conclude, our work focused on developing tools for the real-time monitoring and analysis of the European seabass behavior at sea cages, providing preliminary results from a feeding experiment, to show the tools’ potential. Data on the feeding behavior variations of European seabass are scarce at sea cages. For this, our work has a significant contribution on the field of behavioral monitoring of the Mediterranean species in sea cages. Further data collection of long-time experiments and different feeding parameters is required to enhance the findings of our experiment. Although the system is developed for the European seabass, it is also tested and shows promising results for the gilthead seabream and also salmon. Additionally, we successfully identified distinct behavioral characteristics of the fish under various conditions, including normal, reduced feeding, over-feeding, and fasting. Up to now, we have not achieved full control of the feeding process. Nonetheless, we provided reference values and curves. Moving forward, our next objective is to explore additional feeding scenarios to further enhance the system’s capabilities.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The animal studies were approved by the Ethics Committee of the IMBBC and the relevant veterinary authorities (Ref Number 32257 09-02-2021). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent was obtained from the owners for the participation of their animals in this study.

DG: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. CV: Data curation, Formal analysis, Methodology, Software, Writing – review & editing. NP: Conceptualization, Funding acquisition, Investigation, Project administration, Resources, Supervision, Visualization, Writing – review & editing.