In general, a constrained multi-objective sequence optimization problem involves at least two objectives (Xiang et al., 2020; Wu and Shao, 2024; Shao et al., 2024b; Tian et al., 2024a) and one constraint, which is mathematically formulated as Minimize  f x = f 1 x , f 2 x , … , f m x Subject to  x = x 1 , x 2 , … , x d ∈ Ω g i x ≤ 0 , i = 1 , … , q , (1) where x = ( x 1 , x 2 , … , x d ) represents a d -dimensional decision variable, Ω represents the decision space (i.e., all permutations of 1 , … , d ), f ( x ) denotes its objective vector, m is the dimension of the objective space, and g i ( x ) denotes q inequality constraints. If feasible solutions x and y satisfy f i ( x ) ≤ f i ( y ) for every i ∈ 1 , … , m and f i ( x ) < f i ( y ) for at least one j ∈ 1 , … , m , then x is said to dominate y . The objective in solving Equation 1 is to discover a diversified set of feasible Pareto optimal solutions (Xiong et al., 2024) that are not dominated by any solutions in the decision space Ω (Shao et al., 2023b; Zhang et al., 2024; Jia et al., 2023).

To find multiple feasible and Pareto optimal solutions for constrained multi-objective sequence optimization problems, a number of evolutionary algorithms have been developed in the last decades. In Zhang et al. (2005), the study investigates the multi-job batch flow problem in a two-stage hybrid flow shop. To tackle this NP-hard problem, the authors develop two heuristic methods, both of which involve sorting the jobs first and then applying a strategy of batch flow processing to each job. These two heuristic methods differ in the way they sort the jobs. The first heuristic treats each job as a whole entity. The second heuristic method views the system as a pure flow shop with machine aggregation at the first stage. It uses the summary files of each job from the single job batch flow results as the time requirements for the artificial pure flow shop. When solving the batch flow problem for each job in the sequence, both heuristic methods allocate a balanced number of sub-batches to the machines in the first stage and determine the size of the sub-batches. The results indicate that the aggregated machine heuristic algorithm performs significantly better. The aggregated machine algorithm shows good solution quality, with an average relative distance from the lower bound of only 6.85%. Therefore, it produces high-quality solutions and significantly improves upon the performance of traditional algorithms in this domain.

In qing Li et al. (2020), the authors introduce a heuristic Multi-Objective Evolutionary Algorithm based on Decomposition (MOEA/D) specifically tailored to tackle the complex hybrid flow shop batch scheduling problem. The algorithm makes several significant contributions to optimization in this domain. Firstly, a novel crossover operator is introduced to effectively handle scenarios where parent solutions exhibit varying sub-batch vectors. Secondly, a right-shift heuristic algorithm is proposed, taking into consideration both the problem structure and objective features to enhance the overall performance of the algorithm. Additionally, a population initialization heuristic algorithm is developed, which efficiently allocates each solution to the closest reference vector. Furthermore, a mutation heuristic algorithm is presented, incorporating considerations for sub-block arrangements to enhance the exploitation capabilities of the algorithm. Through rigorous experimentation and testing, the efficacy and efficiency of the proposed algorithm are empirically validated, demonstrating its effectiveness in solving the hybrid flow shop batch scheduling problem.

In Zhang et al. (2022), the study investigates a multi-objective mixed-model assembly line scheduling problem, with the aim of minimizing the maximum completion time and the total number of batches considering setup and transportation operations. A multi-objective mixed integer programming model was established, and a solver was used to evaluate the trade-off between the two objectives. To address this problem, an automatic algorithm design is introduced in the proposed framework to conceptualize an automated multi-objective evolutionary algorithm. This is the first study to use automatic algorithm design to solve a multi-objective mixed-model assembly line scheduling problem. Considering the characteristics of the problem and the algorithm framework, the authors designed configurable settings for numerical parameters and categorical parameters, as well as operators. Subsequently, an automated MOEA was constructed using an iterative racing procedure. Experimental validation of the performance of the proposed algorithm shows its efficiency and effectiveness.

In Duan et al. (2021), to capture the characteristics of real-world vehicle routing applications, the author developed a robust mutlti-objective vehicle routing problem with time windows (RMO-VRPTW), which includes two conflicting objectives: minimizing the number of vehicles and total distance. Additionally, a new form of uncertainty is introduced to capture disruptive features from practical applications. To address RMO-VRPTW, a robust optimization approach was developed, incorporating advanced encoding and decoding methods, robustness measures, and local search strategies. Initially, the deterministic problem space features were thoroughly explored to guide robust optimization. Furthermore, to further explore the search space, two local search strategies were proposed. One adjusts customer priorities based on associated time windows, while the other directly manipulates routes by removing customers from routes with fewer customers and inserting them into routes with stronger robustness.

To solve large-scale car sequence problems, a novel mutation-based multi-objective evolutionary algorithm called MOEA-PGX is proposed in Shao et al. (2023a). The core idea of the MOEA-PGX algorithm lies in extracting heuristic information from the population and constructing a probability matrix based on this information. During the optimization process, this probability matrix is utilized to heuristically repair infeasible solutions while retaining the advantageous genes from the parent solutions. This heuristic repair strategy enhances the quality and feasibility of solutions. To represent solutions, the MOEA-PGX algorithm converts them into permutation groups. By employing permutation-based crossover and mutation operations, high-quality characteristics are maintained when generating offspring solutions. This representation method captures the structural features of sequencing problems better, leading to the generation of superior solutions. Compared to existing algorithms, MOEA-PGX demonstrates faster convergence speed and a lower probability of getting trapped in local optima, making it an effective approach for solving large-scale car sequence problems.

Although the optimization algorithms mentioned above have achieved remarkable performance in various sequential optimization problems, the production scheduling sequence optimization in electric rope shovel production often presents unique challenges (Dong et al., 2024; Xie et al., 2024). Specifically, these algorithms typically rely on designing algorithms based on the characteristics of the problem’s dataset, which are not directly applicable to the optimization scenarios in electric rope shovel production. On the other hand, the sequence of operations in electric rope shovel production is not simply a permutation of 1 , … , d , since multiple components have multiple operations. Additionally, the processing of components in electric rope shovel production often involves specific time requirements, adding strict constraints to the optimization. To address these issues, we propose a cooperative evolutionary algorithm to efficiently solve PSSOPs, the details of which are elaborated in the next section.

Let the set of components (e.g., boom assembly, bucket, etc.) to process be denoted as J = { J 1 , J 2 , … , J n } , where each component J i follows a predefined sequence of operations (e.g., cutting, welding, drilling, painting, etc.) O i 1 , O i 2 , … , O i u i , and O i j denotes the j -th operation of component J i . Given multiple machines W = { W 1 , W 2 , … , W M } , the processing time of operation O i j on machine W k is given by T ijk if the operation can be conducted on W k . For a solution (i.e., the sequence of all operations of all components), the idle waiting time between operations O i ( j − 1 ) and O i j for component J i is E T ijk , and thus the completion time of component J i , denoted by C i , is the total sum of the processing and waiting times of all its operations: C i = ∑ j = 1 u i T ijk + E T ijk . (2)

As for the proposed production scheduling sequence optimization problems (PSSOPs), the core goal is to minimize the maximum completion time across all components, which represents the overall production cycle. This can be formulated as the first objective function: f 1 = max i = 1 , … , n C i , (3)

which ensures that the completion time of the most time-consuming component is minimized, reflecting an optimized production cycle.

While each component should be completed within a specific time window, the second objective function aims to reduce penalties incurred from early or late deliveries. More specifically, if a component is finished earlier than its due time d i 1 , an early completion penalty E i is introduced. Similarly, if a component is completed later than its late due time d i 2 , a tardiness penalty T i is incurred. These penalties are defined as E i = max 0 , d i 1 − C i T i = max 0 , C i − d i 2 , (4)

and the second objective function can be expressed as f 2 = ∑ i = 1 n α E i + β T i . (5) Here, α and β represent the respective weights for early and late penalties, allowing flexibility in balancing the costs of deviation from the scheduled due dates. The setting of these two parameters primarily reflects the different preferences for early and late penalties. Depending on changes in the production environment, these parameters can be adjusted to different combinations. In addition, due to strict time management, the second objective has to be less than a user-given constraint value for all solutions to make sense, so this is a typical constrained multi-objective optimization problem.

As a consequence, the complete definition of PSSOPs is as follows, where S C represents a user-specified parameter for defining the constraint: min f 1 = max i ∈ S C i min f 2 = ∑ i = 1 n α E i + β T i subject to f 2 < S C . (6)

In this optimization model, each solution determines the value of C i for all components. Since a solution should contain the sequence of multiple operations of multiple components, it has to be represented by a complex vector like O 11 , O 31 , O 32 , O 21 , …   . To facilitate the optimization of PSSOPs using various algorithms, we suggest a simple and flexible encoding scheme, representing each solution with a real vector that can be optimized using most constrained multi-objective evolutionary algorithms. As illustrated in Figure 1, we demonstrate a processing task involving three components: Component 1 has three operation, Component 2 has three operations, and Component 3 has four operations. The example solution is a real-coded vector (0.10,0.42,0.58,0.15,0.29,0.81,0.23,0.36,0.77,0.93), where each dimension corresponds to an ordered operation. To obtain the sequence of all operations, the solution is decoded by sorting all its real elements in an ascending order. Then, the resulting permutation (1,4,7,5,8,2,3,9,6,10) is converted into a sequence of operations, where elements 1,2,3 correspond to the three operations of Component 1, elements 4,5,6 correspond to the three operations of Component 2, and elements 7,8,9,10 correspond to the four operations of Component 3. Note that the rank of all operations of a component is predefined and cannot be modified, hence the elements corresponding to a component do not need to be associated with specific operations. Lastly, to calculate the completion time C i , the operations are conducted one by one on specific machines, and they should be waited if other operations are being conducted on the same machine.

With the above encoding scheme, the proposed PSSOPs turn out to be continuous constrained multi-objective optimization problems, which can be handled by many constrained multi-objective evolutionary algorithms in theory. However, the conflicting objectives and strict constraints challenge many existing algorithms in finding feasible Pareto optimal solutions, especially when the landscape is still highly discretized due to the conversions from real vectors to discrete sequences. Therefore, an effective evolutionary algorithm is tailored for solving PSSOPs, the details of which are presented in the next subsection.

The procedure of the proposed neo-cooperation search based evolutionary algorithm (NCSEA) is illustrated in Algorithm 1, which begins with the initialization of neural network and two populations, P o p u l a t i o n 1 and P o p u l a t i o n 2 , both of which are created randomly (Lines 1, 3 and 4). The Neural network N e t are used to learn the optimal variation granularity a c t i o n . The proposed algorithm also sets the initial count of consumed evaluations, F E , equal to the size of the populations (Line 6). Once the populations are established, the proposed algorithm enters a loop that continues until the maximum number of evaluations, F E max , is reached. Within this loop, the proposed algorithm randomly selects N parents from P o p u l a t i o n 1 and randomly selects N parents from P o p u l a t i o n 2 (Lines 8 and 9). This selection process is used for maintaining a good performance gene pool, which enhances the algorithm’s ability to explore various solutions. After selecting the parents, the proposed algorithm generates N offspring solutions from each set of parents (Lines 12 and 13). These offspring solutions represent new potential solutions that will be introduced to the populations. Since the optimization model of the proposed PSSOP introduces a novel encoding scheme representing solutions with real vectors, the real variation operators used in many evolutionary algorithms can be adopted, where the simulated binary crossover and polynomial mutation are adopted in the proposed NCSEA. Given two parent solutions x 1 and x 2 , two offspring solutions o 1 and o 2 are generated by simulated binary crossover (Deb and Agrawal, 1995) and polynomial mutation (Deb and Goyal, 1996). Following the generation of offspring solutions, the proposed algorithm updates both P o p u l a t i o n 1 and P o p u l a t i o n 2 by combining each current population with all the newly created offspring solutions (Lines 12 and 13). This combination is designed to foster diversity and improve the quality of solutions.

Algorithm 1. Main procedure of NCSEA.

The next step, i.e., environmental selection, involves retaining N solutions from P o p u l a t i o n 1 and retaining N solutions from P o p u l a t i o n 2 (Lines 14 and 15), where the strategies are different for retaining solutions from the two populations. P o p u l a t i o n 1 undergoes a selection process known as environmental selection, where N solutions are chosen based on their performance according to non-dominated sorting and crowding distances, on the basis of constrained Pareto dominance relations. More specifically, the constraint violation of a solution x is calculated by C V x = max 0 , f 2 x − S C , (7)

so that a smaller C V ( x ) indicates a smaller constraint violation, and x is feasible if and only if C V ( x ) = 0 . Then, a solution x is said to be better than (i.e., constrained dominate) another solution y if and only if C V x < C V y , (8) or C V x = C V y f 1 x ≤ f 1 y  and  f 2 x ≤ f 2 y f 1 x ≠ f 1 y  or  f 2 x ≠ f 2 y , (9)

On the other hand, P o p u l a t i o n 2 is truncated based on only the second objective, the minimization of which is also beneficial for the satisfaction of the constraint. More specifically, the N solutions with smaller f 2 values are retained in P o p u l a t i o n 2 . That is, a solution x is said to be better than (i.e., constrained dominate) another solution y if and only if f 2 x < f 2 y . (10)

At the end of each loop, the proposed algorithm updates the agent as illustrated in Algorithm 2 (Line 17). The optimal mutation granularity a c t i o n of the next iteration is predicted by Algorithm 3 (Line 18). The evaluation count, F E , is updated to reflect the number of evaluations consumed during that iteration (Lnie 19). This ensures that the algorithm stays within the specified limits of function evaluations. Finally, once the loop completes and the maximum evaluation count is reached, the proposed algorithm returns the final population P o p u l a t i o n 1 (Line 20), which contains the most promising solutions discovered throughout the process. It is worth noting that since the second objective involves constraints, the main purpose of the second population is to handle the second objective by ensuring that the constraints are satisfied. This structured approach allows NCSEA to efficiently explore and exploit the solution space, ultimately leading to high-quality outcomes.

Algorithm 2. T r a i n i n g D Q N ( N e t , M ) .

Algorithm 3. A d a p t i v e M u t a t i o n ( N e t , M ) .

To further improve the algorithm’s exploration ability, reinforcement learning is employed to determine the optimal mutation granularity as illustrated in Algorithm 2 and Algorithm 3. Specifically, five mutation granularities— 1 d , 1 2 d , 3 2 d , 4 2 d and 1 5 d (where d represents the decision variable dimension)–are used as candidate strategies to effectively balance exploration and exploitation. The population state is defined by convergence, diversity, and feasibility metrics. Convergence is evaluated by the population’s average performance on each objective function, indicating how close solutions are to the Pareto optimal front. For each objective function f i ( x ) , the convergence is calculated as follows: c o n = ∑ i = 1 m ∑ x ∈ P f i x , (11) where f i ( x ) represents the value of the i -th objective function, and m is the number of objective functions. Diversity captures the population’s spread across each objective function, which is calculated using: d i v = ∑ i = 1 m ∑ x ∈ P f i x − o b j i 2 , (12) where o b j i is the average value of the i -th objective function. While feasibility measures the degree to which the population satisfies problem constraints; a value of 0 denotes complete feasibility, while higher values suggest constraint violations. Feasibility is defined as: f e a = ∑ x ∈ P C V x , (13) where C V ( x ) denotes the constraint violation of solution x .

Together, these elements—convergence, diversity, and feasibility—form the population state s t , encapsulating key attributes that allow the reinforcement learning agent to suggest the ideal mutation granularity. The hypervolume (HV) metric serves as the reward signal r t to assess resource allocation effectiveness, measuring convergence and diversity by evaluating the solution set’s enclosed volume. Each training entry comprises the current state, action, obtained reward, and the new state. To be specific, the states is a three-dimensional vector. The reward and action are a scalar, respectively. These data are generated at each iteration and sequentially inserted into the experience memory pool. This experience pool continuously enhances the agent’s decision-making capabilities. The agent is update rule according to: Q s , a ; θ ← Q s , a ; θ + α y − Q s , a ; θ (14) where α is the learning rate, θ are the network parameters, and the target y is defined as: y = r + γ max a ′ Q s ′ , a ′ ; θ − (15) Here, r is the reward received after action a , s ′ is the next state, and θ − represents the parameters of the target network. The DQN network architecture comprises an input layer with 4 nodes, two hidden layers, and an output layer. The input layer receives a 4-dimensional state vector as input. This is followed by the first hidden layer, which contains 10 nodes with a nonlinear activation function (i.e., ReLU) to enhance learning. The second hidden layer also processes information in preparation for the output layer, which has a single node that provides the Q-value for a particular action-state pair. This configuration enables the network to learn an effective mapping from states to action values in a compact, efficient structure. Based on the established mapping relationship, the reinforcement learning agent can recommend the optimal mutation granularity for the current population during the iteration process, i.e., determining the parameters for polynomial mutation that are suitable for the current population, thereby guiding the generation of offspring.

From the above description, it can be seen that the proposed algorithm considers all the optimization objectives and constraint through the first population, while the second population focuses solely on the second objective. For the second population, since it only selects solutions that perform significantly on the second objective to generate offspring solutions, it is more likely to excel in the second objective. Moreover, if only the second population is used, the entire population will struggle to address the first objective, which is why the first population focuses on all the optimization objectives and constraint. It is worth noting that the offspring solutions generated by both populations are shared, allowing the second population to adaptively adjust its search for the second objective using the offspring solutions generated by the first population.

The coevolution mechanism of the proposed NCSEA is different from existing co-evolutionary algorithms for constrained multi-objective optimization. To be specific, most existing algorithms evolve a main population considering all objectives and constraints of the problem, and evolve one or more auxiliary populations eliminating part or all of the constraints. Such coevolution mechanism can help the main population to jump over local feasible regions, but is not effective enough for the proposed PSSOPs with highly discretized landscapes that are difficult to converge. On the contrary, the proposed NCSEA suggests a problem-dependent coevolution mechanism considering part of the objectives in an auxiliary population, which exhibits significantly better performance than existing algorithms as evidenced by the experimental results given in the next section.

Six datasets with different conditions for electric rope shovel production are involved in the experiments, where there are a total of 14 components, each of which requires 3 to 6 operations to complete on one of eight machines within specific processing durations. For instance, in the production of electric rope shovels, the processing of the boom assembly requires five operations, including cutting, welding, drilling, heat treatment, and painting, to ensure strength and precision. The processing of the bucket requires six operations, including steel plate cutting, forming, welding, heat treatment, surface treatment, and wear-resistant coating, to enhance durability and abrasion resistance. The processing of the stick involves four operations, including cutting, welding, drilling, and painting, to ensure a precise fit with other components. As a result, the experiments involve six test instances denoted as PSSOP1–PSSOP6, each having 62 operations with different processing times and time windows. Specifically, each operation in PSSOP1 and PSSOP2 has a longer operation time, each operation in PSSOP3 and PSSOP4 involves more machines, and the time windows in PSSOP5 and PSSOP6 are more restricted. Due to these differing characteristics, these PSSOPs pose challenges for CMOEAs. Besides, the parameters α and β for early and late penalties are set to 0.8 and 0.2, respectively, which can prefer handling of early penalties. The size of the experience pool is set to 1 000 to ensure there is enough space to store the experiences.

The proposed algorithm in this study is compared with six state-of-the-art constrained multi-objective evolutionary algorithms: TriP (Ming et al., 2022), EMCMO (Qiao et al., 2022), CMOQLMT (Ming et al., 2023), CMOSMA (He et al., 2022), DP-PPS (Ming et al., 2022), and C3M (Sun et al., 2022). For fair comparisons, compared algorithms follow the parameter settings in their original papers and all of them use simulated binary crossover and polynomial mutation to generate real-coded offspring solutions for PSSOPs, where the parameter η is set to 20 and the mutation probability p r o b is set to 1 / d ( d is the number of decision variables). Each algorithm uses a population size of 100 and undergoes 10,000 function evaluations, resulting in each optimization process lasting tens of minutes for each test instance. At the same time, the function evaluation setting is large enough to detect the performance of each comparison algorithm and the proposed algorithm. As the Pareto fronts of the optimization problem are unknown, the hypervolume (HV) indicator is employed to evaluate the quality of each solution set. To ensure result reliability, 30 independent runs are conducted for each algorithm on every test instance, followed by a Wilcoxon rank sum test. Detailed experimental evidence is provided in the subsequent subsection to showcase the superior performance of the proposed algorithm.

The optimization results of the proposed algorithm and four comparative algorithms on PSSOP1–PSSOP6 are presented in Table 1. It can be observed that the proposed algorithm performs the best on all the six test instances, which means that the proposed algorithm significantly outperforms TriP, EMCMO, CMOQLMT, CMOSMA, DP-PPS, and C3M on PSSOPs. Moreover, Figure 2 displays the convergence curves of their HV values on PSSOP1–PSSOP6. The plots indicate that the proposed algorithm converges faster than the compared algorithms TriP, EMCMO, CMOQLMT, CMOSMA, DP-PPS, and C3M. It is worth noting that even with only 6,000 function evaluations, the population generated by the proposed algorithm can compete with those generated byTriP, EMCMO, CMOQLMT, CMOSMA, DP-PPS, and C3M, which have undergone 10,000 function evaluations on these test instances. To provide a more intuitive demonstration of the optimization results, Figure 3 shows the objective values of the final populations on PSSOP2 and PSSOP5. It can be observed that the proposed algorithm gains solutions dominating the solutions obtained by TriP, EMCMO, CMOQLMT, CMOSMA, DP-PPS, and C3M, further confirming the superiority of the proposed algorithm. It is worth noting that the proposed algorithm significantly outperforms the comparison algorithms in both optimization objectives. This indicates that the sequence solution found by the proposed algorithm can not only produce the corresponding parts within the specified time period, but also accelerate the entire production process, offering advantages in improving production efficiency and reducing costs.

To further validate the effectiveness of the proposed collaborative search method, NCSEA was compared with its variants that use a single search scheme, thereby completely eliminating the impact of other strategy differences. Table 2 lists the comparison results of NCSEA and its two variants, where NCSEA1 uses only population1, i.e., it only performs global search, and NCSEA2 uses only population2, i.e., it only performs local search. Clearly, the proposed NCSEA still demonstrates the best overall performance and is competitive with the different variants of NCSEA.

Furthermore, a comprehensive assessment of the computational efficiency of the seven compared algorithms is presented. As depicted in Table 3, an in-depth breakdown of the average runtime across TriP, EMCMO, CMOQLMT, CMOSMA, DP-PPS, C3M, and the proposed NCSEA is provided. Upon meticulous data analysis, it becomes evident that the proposed algorithm demonstrates competitive computational efficiency when compared with other algorithms. This observation underscores the robust computational efficiency of NCSEA, a purpose-built algorithm tailored to efficiently address optimization challenges brought by electric rope shovel production scheduling. Consequently, the NCSEA presented in this study emerges as a highly efficient algorithm for PSSOPs.