Modern LLMs are predominantly transformer-based (Vaswani et al., 2023) and achieve strong performance on general-purpose tasks via pretraining on large datasets. The text-to-text framing method introduced by the T5 family of encoder-decoder models (Figure 1) provides a sequence-to-sequence translation process for a wide range of downstream tasks (Raffel et al., 2023). Building on this foundation, instruction tuning has been shown to substantially improve zero-shot generalization for T5 models, including FLAN-T5 (Wei et al., 2022). These instruction-tuned sequence-to-sequence models are therefore a practical starting point for domain-specific text generation tasks, particularly when the target can be expressed as a structured mapping from an input text prompt to an output text sequence.

However, general-purpose pretraining does not guarantee reliable performance in specialized domains. Empirical studies have documented failures such as confident but incorrect model outputs, described as hallucinations (Zhang et al., 2023), and poor performance on domain-specific tasks when relevant knowledge is underrepresented in pretraining data (Nori et al., 2023). More broadly, domain and task mismatch has motivated continued emphasis on model adaptation methods, including domain-specific pretraining and fine-tuning, as ways to align model behavior with a target downstream task (Gururangan et al., 2020; Tinn et al., 2023).

Full-precision fine-tuning adapts a pretrained model by updating a large fraction of parameters, often yielding strong downstream performance but incurring substantial computational and memory costs for transformer-scale models. These compute and memory costs arise from storing model parameters, gradients, and optimizer states and the distributed training overhead that scales with model size and sequence length. Large-scale optimization techniques such as ZeRO reduce memory redundancy and improve feasibility for large model training, but practical constraints remain for resource-limited environments (Rajbhandari et al., 2020).

Parameter-efficient fine-tuning (PEFT) approaches reduce fine-tuning compute and memory costs by restricting trainable parameters. PEFT approaches include LoRA (Low-Rank Adaptation) for injecting low-rank matrices, adapters for adding small bottleneck layers, prefix tuning/prompt tuning for learning continuous prompts, and QLoRA/DoRA for enhanced efficiency (quantization/decomposition). Herein, the PEFT methods LoRA and QLoRA are compared with full-precision fine-tuning. LoRA injects low-rank trainable adapter weight matrices into targeted model layers while freezing the base weights, enabling competitive downstream performance with fewer trainable parameters than the full-precision method (Hu et al., 2021). QLoRA extends this regime by quantizing the base model weights while training adapters weight matrices, improving memory efficiency and enabling fine-tuning of larger models under higher compute constraints (Dettmers et al., 2023). Surveys and unifying perspectives on PEFT further emphasize that adapter-based methods can provide strong performance-efficiency tradeoffs across tasks and model classes (Han et al., 2024; He et al., 2022).

Knowledge distillation is the transferring of capability from a larger teacher model to a smaller student model by using teacher-produced targets as supervision for the student (Hinton et al., 2015). Distillation Step-by-Step (DSS) augments this paradigm by training on both the targets and intermediate rationales produced by the teacher model, typically elicited through chain-of-thought prompting (Wei et al., 2023; Hsieh et al., 2023). DSS has been shown to improve data efficiency on benchmark reasoning tasks, enabling smaller models to approach or exceed the performance of larger models trained with conventional supervision under comparable compute and dataset constraints (Hsieh et al., 2023).

However, the extent to which rationale supervision improves downstream performance may be task dependent. This open question motivates ablation studies that compare rationale-augmented training against label-only supervision under controlled training conditions, especially when combined with PEFT and quantized optimization regimes where model capacity and optimization dynamics differ from full-precision training (Hu et al., 2021; Dettmers et al., 2023).

Mapping natural language questions to executable structured queries is a classic semantic parsing problem, widely studied in text-to-SQL and related benchmarks. Large-scale datasets such as Spider emphasize cross-domain generalization and complex compositional query generation (Yu et al., 2018), while earlier approaches established sequence-to-sequence structured query generation as a viable modeling strategy (Zhong et al., 2017). A persistent practical challenge in this area is output validity: syntactically invalid structured outputs are unusable even when semantically close to the target. Constrained decoding methods such as PICARD explicitly enforce admissible structure during autoregressive decoding and have been shown to improve validity and performance with models such as T5 (Scholak et al., 2021).

In operational systems, structured query representations also extend beyond SQL to domain-specific query languages. Query DSL, as used in OpenSearch, is one such representation and is commonly expressed as JSON (Contributors, 2024). This motivates evaluation setups that consider both semantic correctness and structural validity when fine-tuning LLMs for natural-language to Query-DSL translation.

Building on instruction-tuned sequence-to-sequence modeling (Wei et al., 2022; Raffel et al., 2023), PEFT methods (LoRA/QLoRA) (Hu et al., 2021; Dettmers et al., 2023), and DSS rationale-augmented fine-tuning (Hsieh et al., 2023), this work proposes and evaluates a strategy for constructing a domain-specific dataset via DSS and benchmarks full-precision fine-tuning against LoRA and QLoRA under realistic resource constraints. Additionally, we include an ablation study comparing rationale-augmented training to label-only supervision to isolate the contribution of rationale supervision within this task setting.

Distilling Step-by-Step is a method by which a teacher LLM is used to generate feature labels and label rationales in order to train a smaller model to match the teacher model's performance on the task. DSS fine-tuned models have been shown to achieve superior performance, using smaller fine-tuning datasets, than larger models trained with standard fine-tuning methods on benchmark datasets and on few-shot learning tasks (Hsieh et al., 2023).

The generation of label rationales was done by prompting teacher models using chain-of-thought (Wei et al., 2023). This technique provided intermediate rationales to explain the connections between the input and the label (Hsieh et al., 2023). An example input prompt is shown in Figure 2. The prompt consists of three distinct parts that are designed to provide information on the DSL interface and dataset of the downstream task while inducing a chain-of-thought reasoning process in the model. The final section, **Question**, contains an example input question from the user. Using this prompt schema, the rationals and DSL JSON output labels were extracted from the teacher model.

Once rationales and labels were extracted for all inputs, the rationales were incorporated into the fine-tuning dataset by setting up the model fine-tuning as a multi-task problem. For each input, a task prefix was prepended to the input features. Each task prefix corresponds to the resulting output (label or rationale) and enables the student model to differentiate between each task (label generation or rationale generation). Thus, the student model was trained to generate the correct rationales and labels for each input.

This work utilized a fine-tuning dataset designed around the conversion of natural language questions to Query Domain-Specific Language, or Query DSL, for use by practitioners to query organizational data without requiring knowledge of database querying languages. Fine-tuning a language model to perform this translation task would enable practitioners to interact with databases in a more conversational way, leading to broader use. Query DSL is a flexible search language used by the OpenSearch data-analysis platform to parse through databases (Contributors, 2024). The fine-tuning dataset input features consisted of 1000 natural language questions to be translated to Query DSL for searching through the OpenSearch database. Each of these input questions was matched to an output label that consisted of a properly formatted Query DSL JavaScript Object Notation (JSON) object that provided the desired OpenSearch results based upon the input question and an output rationale that described how to generate the corresponding Query DSL JSON in a concise, step-by-step fashion. Figure 3 gives an example rationale and label generation process for a given input. Figure 4 shows an example training step using labels and rationales. Both the Query DSL output labels and rationales were generated as described in the previous subsection, using the open-source model Mixtral 8x22B, a Sparse Mixture of Experts (MOE) LLM (Team, 2024) as the teacher model. In total, the dataset consisted of 1000 input questions, along with 1000 output labels and rationale pairs. The full dataset was used for fine-tuning.

The multiple architectures of the FLAN T5 model that were used in this work are shown in Table 1. There are several key differences between each model variant. Most notably, the total number of parameters is governed by the number of encoder and decoder layers, the dimensionality (i.e., size) of the feedforward layers, and the number of attention heads per attention block within each encoder and decoder layer. Between the FLAN T5 Small, T5 Base, and T5 Large, there is a consistent increase in the dimensionality of the feedforward layers, the number of encoder and decoder layers, and the number of attention heads, which results in a consistent increase in the total number of parameters as the model size increases. However, the FLAN T5 XL has feedforward layers with four times as many parameters as the FLAN T5 Large and twice as many attention heads per block, resulting in a large increase in the total number of parameters in the FLAN T5 XL over the FLAN T5 Large. In this work, we utilized FLAN T5 Small, Base, Large, and XL architectures to determine the performance and efficiency of the three fine-tuning strategies (full-precision, LoRA, and QLoRA) using the Query DSL dataset.

Fine-tuning is the process by which a pre-trained language model is trained on additional tasks using a smaller dataset than the pre-training corpus. The classical fine-tuning method is to make some or all of the language model parameters available for training using full floating-point precision model weights, then train the model weights using a lower learning rate than the learning rate used during pre-training. In this work, this classical fine-tuning method is referred to as full-precision fine-tuning. Although the full-precision method can be very effective in terms of performance, it comes at a high computational cost, especially in terms of GPU memory consumption (Quentin and Stella Biderman, 2023). Due to this high cost, efficient methods have been developed to provide a balance between cost and performance. Two such methods, along with full-precision fine-tuning, were investigated: LoRA and QLoRA fine-tuning (see Figure 5).

LoRA fine-tuning, originally demonstrated in Hu et al. (2021), freezes the pre-trained model weights and injects trainable rank decomposition matrices into specified layers of the transformer architecture, which, due to reducing the number of trainable parameters for downstream tasks, theoretically reduces the computational cost of fine-tuning the model. As shown in Hu et al. (2021), a GPT-3 175B model fine-tuned with the Adam optimizer and LoRA fine-tuning reduced the number of trainable parameters by 10,000 times and the GPU memory requirements by 3 times while performing on par or better than standard fine-tuning on several benchmarks (Hu et al., 2021).

QLoRA fine-tuning is designed for further reductions in computational cost without sacrificing model performance. Originally shown in Dettmers et al. (2023), QLoRA quantizes the pre-trained model weights to 4-bit NormalFloat, a theoretically optimal quantization data type (Dettmers et al., 2023) and utilizes paged optimizers to avoid memory spikes that occur when processing mini-batches with long sequence lengths (Dettmers et al., 2023). QLoRA has been shown to match full-precision fine-tuning and LoRA fine-tuning performance on multiple benchmarks (Dettmers et al., 2023).

To provide scalable distributed computing capabilities for model training, a two-node, GPU-enabled Rancher 2.0 Community Edition (Rancher, 2024) Kubernetes cluster was utilized for the hyperparameter search. The compute nodes were provisioned with two Intel Xeon Platinum 8480+ processors, 2 TB RAM, four NVIDIA 80 GB H100 GPUs, and non-volatile memory express (NVME) solid-state storage.

Each variant of FLAN T5 was trained on the Query DSL dataset developed using DSS. Fine-tuning jobs were submitted using the Ray Train configured for Distributed Data Processing through the Accelerator API (Team, 2023).

The fine-tuning dataset was split in an 80/20 fashion into training and evaluation datasets, and all hyperparameter sweeps utilized the same split. The models were trained for 100 epochs with evaluation taking place after each epoch, and the evaluation loss was monitored in order to conduct learning rate reduction after 10 epochs of no improvement. Loss was calculated in accordance with (Hsieh et al. 2023):

where L_label is the label prediction loss and l is the cross-entropy loss between the predicted tokens x_i and label tokens y_i. Additionally, we framed the learning as a multi-task problem in accordance with Hsieh et al. (2023), where we train the model to predict not only the task labels but also the rationales given an input:

where L_total is the combined loss between L_label, described above, with L_rationale

this is the cross-entropy loss between the predicted tokens x_i and the rationale for the label r_i. In Equation 2, α was used as an additional training hyperparameter to tune how much loss the model accumulates from predicting rationales. For this work, α was set to 0.5 to balance the loss between predicted rationales and predicted labels.

The hyperparameters that were utilized across all of the hyperparameter sweeps are shown in Table 2. The learning rate was based upon optimal results found in Hsieh et al. (2023) for the T5 neural network architectures, and the batch size was given as the number of training samples per batch across all nodes. In this work, one training sample per batch for each of the 8 nodes in the cluster results in a total batch size of 8 training samples per batch. As shown in Equation 2, Alpha was set to 0.5 to balance the loss resulting from rationales and labels.

Selected LoRA and QLoRA-specific hyperparameters (Target Modules and Dropout Rate) were held consistent throughout the hyperparameter sweeps for both methods. The target modules were selected based upon performance detailed in Dettmers et al. (2023), and dropout rate (Srivastava et al., 2014) was selected based upon performance considerations given in Lin et al. (2024). The other hyperparameter spaces searched during this work were the LoRA and QLoRA Rank and Alpha hyperparameters. The Rank hyperparameter determines the dimensionality of the LoRA/QLoRA matrices (Hu et al., 2021), and the Alpha hyperparameter serves as a scaling factor for the updates made by the LoRA/QLoRA matrices to the original model weights (Hu et al., 2021). For a given value of Rank, a series of Alpha hyperparameters were tested. The Rank values tested in this work were 32, 64, and 128. Alpha values paired with Rank 32 were 32, 64, and 128. Alpha values paired with Rank 64 were 32, 64, 128, and 256. Alpha values paired with Rank 128 were 32, 64, 128, 256, and 512. In this way, Alpha to Rank ratios of 1:4, 1:2, 1:1, 2:1, 3:1, and 4:1 were represented in the hyperparameter search to determine if an optimal Alpha to Rank ratio exists.

The hyperparameter search consisted of 86 hyperparameter sweeps spread across all three methods and for the four model types examined. For a given set of hyperparameters, a fine-tuning run for each model architecture and fine-tuning method was attempted, beginning with the smallest model. If the run was successful, the next model architecture, in order of increasing size, was utilized for the next run attempt. This process was continued for each successive model architecture until the models would not run due to GPU memory limitations on the cluster. After this failure point was reached, the next fine-tuning method would be selected, and the process would be repeated with the same hyperparameters, beginning with the smallest model. When all fine-tuning methods and model architecture sizes were attempted for a given set of hyperparameters, the hyperparameter set would be changed, and the entire process would begin anew. In total, 3 runs were performed using full-precision fine-tuning, 39 runs were performed with the LoRA fine-tuning method, and 44 runs with the QLoRA fine-tuning method. Twenty-nine hyperparameter sweeps were performed using the FLAN T5 Small model architecture, 29 using the FLAN T5 Base architecture, 29 using the FLAN T5 Large architecture, and 5 sweeps using the FLAN T5 XL architecture. Due to GPU memory usage constraints, all of the FLAN T5 XL runs were executed using the QLoRA fine-tuning method. The total compute time for the hyperparameter search was 499.6 hours.

To quantify the contribution of DSS rationales to the performance of the downstream task, we performed an ablation comparing two variants of the loss function described in Equation 2: rational-augmented supervision (Alpha = 0.5), where both label and rationale predictions contribute equally to the total loss, and label-only supervision (Alpha = 1.0), where loss due to rationales is ignored during optimization. All other hyperparameters were kept constant throughout to ensure that differences in performance could be attributed exclusively to the presence or absence of rationale supervision.

The ablation spanned all three fine-tuning modalities and three model sizes: FLAN-T5 Small, Base, and Large. These models were selected to capture the variation in the model size while allowing all modalities to be compared, given the computational constraints. Each ablation run used the same data set and train-validation split as the full experiments, and the results were averaged over two random seeds to reduce the variance due to initialization effects.

For the final evaluation, the best evaluation loss achieved for each hyperparameter sweep was used to perform the final comparison. The fine-tuned student models across all modalities were tested using the same evaluation data. Additionally, the computational cost of each fine-tuning method was compared by using training samples per second and total training time metrics for each model and fine-tuning modality that were captured by the Rancher Kubernetes Cluster.

The results shown in Table 3 demonstrate clear trade-offs between evaluation performance and GPU memory consumption in fine-tuning methods and model architectures. Although the full-precision fine-tuning method consistently achieved better evaluation loss across the model architectures, with the exception of the FLAN-T5 XL, where resource limitations prevented its employment, it did so at the cost of significantly higher average GPU memory usage for the smaller model architectures (FLAN-T5 Small and FLAN-T5 Base). This result highlights the inherent challenge of employing the full-precision fine-tuning method in resource-constrained environments, where GPU memory is often limited. These GPU memory limitations often prevent, and in this case, did prevent, the application of larger and potentially more capable model architectures to specific problems in environments where GPU memory is a finite resource (Figure 6). This GPU memory cost, however, is offset by the smaller number of training runs required to find the hyperparameter combination that yielded better evaluation performance than the other methods. This offset suggests that while GPU memory-based costs may be a higher up-front cost associated with full-precision fine-tuning, the lower number of runs results in a cheaper total hyperparameter search in terms of total compute time. This demonstrates that for environments where GPU compute capacity is not a constraint and/or environments where time-based usage is a constraint, then full-precision is the most efficient and appropriate method. In practical terms, these findings support a decision framework for selecting a fine-tuning method based on deployment constraints. When peak task performance and shortest model-development cycles are the primary objective, full-precision is the most effective choice. When memory availability is the dominant constraint and model size must be maximized within fixed GPU memory, QLoRA is a suitable option. When both time and GPU memory are constraints, LoRA is the optimal choice. Rather than a single optimal method, the results indicate that environments with varied constraints benefit from different fine-tuning methodology selections.

When GPU resources are a primary constraint, the LoRA and QLoRA methods provide memory-efficient alternatives to full-precision fine-tuning for the smaller model architectures like FLAN-T5 Small and FLAN-T5 Base based upon the comparison of evaluation loss performance and memory consumption. When combined with DSS, which reduces the effort required to generate a high-quality dataset for fine-tuning, these methods offer practical solutions for real-world fine-tuning applications where GPU resources are the primary constraint. By reducing dataset creation time, dataset complexity, and memory consumption simultaneously, this pipeline (DSS plus LoRA/QLoRA fine-tuning) provides an opportunity for rapidly deploying fine-tuned, task-specific, language models on memory-constrained devices or smaller cloud configurations.

The memory efficiency of these fine-tuning methods became less evident as model size increased. The LoRA and QLoRA methods required significantly more memory, on average, than the full-precision method when using the FLAN-T5 Large architecture, but the QLoRA method was the only one that enabled the fine-tuning of the FLAN-T5 XL model with the available resources. Given this, the QLoRA method provides an avenue to fine-tune large model architectures on smaller compute clusters and devices at a reasonable performance trade-off to the full-precision method. DSS complements this scalability by enabling the efficient creation of datasets optimized for model fine-tuning, further reducing the total time required to achieve strong performance on domain-specific tasks. In practice, employing the outlined strategy using QLoRA will enable the fine-tuning of large model architectures in memory-constrained environments.

The marginal improvement in QLoRA's training samples per second was observed for the FLAN-T5 XL architecture over the FLAN-T5 Large architecture (Figure 4). This shows that QLoRA's theoretical optimizations over the LoRA method, such as its quantization and paged-memory strategies, provide real-world advantages in computational cost for fine-tuning larger models. This efficiency advantage in samples per second is contrasted by the increased mean training time per run for the QLoRA method. The advantages of requiring less compute are offset by utilizing those resources for longer periods of time.

With regards to the scalability of the hyperparameter search, full-precision fine-tuning included searching over 7 hyperparameters (Table 2) that were baselined according to previous research and only required 3 runs to achieve strong performance. The LoRA and QLoRA methods required searching over an additional 4 hyperparameters (Rank, Alpha, Target Modules, Dropout Rate) that required a large number of runs to converge upon values that enabled comparable performance to the full-precision method.

The analysis of LoRA and QLoRA fine-tuned models revealed a strong correlation between evaluation performance and the values of the LoRA Rank and Alpha hyperparameters. Higher Rank and Alpha values led to improved average evaluation loss, with peak performance observed at ratios of 4:1 for Alpha to Rank. This suggests that increasing both parameters enables the model to learn richer representations without significantly impacting computational efficiency. It is important to note that while this work did not examine Alpha and Rank values that would exceed that 4:1 ratio, there was no indication in the data that increasing Rank and Alpha further would decrease model performance or significantly decrease the computational efficiency of fine-tuning. This suggests that further increases in Rank and Alpha values, and larger ratios of Alpha to Rank, may provide additional boosts to fine-tuning performance. For this reason, this work recommends using a LoRA Alpha to Rank ratio of at least 4:1, with the highest values for Alpha and Rank feasible, as a starting point for hyperparameter searches using these methods.

The ablation results in Table 4 were designed to isolate the effect of DSS rationales by comparing training with rationales included (α = 0.5) with training with only labels (α = 1.0) across multiple model sizes and fine-tuning methods. Across the eight configurations examined, training with included DSS rationales yielded lower evaluation loss than label-only training, demonstrating that DSS training consistently improves model fine-tuning on the Query DSL translation task.

The magnitude of the improvement varied as a function of model size and fine-tuning strategy. The largest reduction in evaluation loss occurred in settings where model size and update precision were most limited: FLAN-T5 Small and, in particular, FLAN-T5 Small using QLoRA. In contrast, the smallest performance differences were observed for larger model sizes with higher precision, suggesting that the larger models were able to learn much of the task structure from the prompt and output labels alone. Taken together, these results indicate that rationale supervision acts as a structured input signal that improves learning efficiency for all models and is more impactful when computational constraints impose limits on the size of the student model. Even for larger models, where improvements were smaller, rationales continued to provide measurable improvement, indicating a comprehensive added benefit rather than an effect limited to small model architectures.

These observations align with prior work demonstrating that rationales improve learning efficiency by imposing intermediate reasoning that structures model learning (Hsieh et al., 2023). The present study extends those findings by showing that rationale supervision remains beneficial even when paired with parameter-efficient fine-tuning and quantized model weights on a more complex task, variables that were not present in earlier DSS research. This suggests that DSS rationales are not only a mechanism for improving few-shot generalization but also a tool for accelerating model learning convergence in resource-constrained fine-tuning pipelines.

Among the fine-tuning methods, full-precision is shown to be the highest-performing and most computationally efficient method in terms of total time to converge on optimal hyperparameters. These attributes make the combination of DSS and full-precision fine-tuning the most attractive choice for a wide range of applications, particularly in scenarios requiring rapid development cycles or constrained computational budgets. The combination of DSS with the LoRA and QLoRA methods, while less effective in terms of evaluation loss than full-precision, exhibited more efficient GPU memory usage for most model architectures and enabled the training of larger models at the cost of longer training times and a longer total hyperparameter search. These observations make LoRA and QLoRA fine-tuning methods practical options for tasks where memory constraints are a primary consideration and/or where lengthy hyperparameter searches may not be a concern, such as edge deployments, GPU-constrained systems, or limited cloud environments.

Taken together, the performance benchmarking, scalability assessment, and ablation findings indicate that efficiency in dataset construction and efficiency in model fine-tuning accelerate the application of fine-tuned models to specific tasks. DSS reduces the data requirements across all fine-tuning methods, while the constraints of a given resource environment inform the choice between full-precision, LoRA, and QLoRA without altering the underlying training pipeline. From this, the approach introduced in this study can be applied as a modular framework where DSS serves as a consistent foundation for efficient dataset creation, and the fine-tuning method can be selected based on available compute resources, model size, and operational constraints. As a result, the choice of fine-tuning method becomes a deployment decision rather than a performance constraint.

Several limitations should be considered when interpreting this work and can be considered directions for future study. First, the empirical evaluation is confined to a single downstream task: translation from natural language questions to Query DSL for a specific downstream data environment. Although this task is representative of a broader family of natural language to formal language translation problems, the conclusions may not directly transfer to other domains, such as open-ended text generation, conversational dialogue, or classification tasks without further validation. Second, the evaluation metrics used in this study are limited. The primary metric is token-level evaluation loss on a held-out validation set, supplemented by computational metrics such as training samples per second, total training time, and GPU memory usage. While useful for characterizing training dynamics and resource consumption, and while all training examples were screened for correctness and validity, the metrics do not directly capture task-level correctness, such as exact match rates on the DSL JSON. Although the ablation shows consistent gains in loss when rationales are included, future work should incorporate additional metrics, such as Bilingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit Ordering (METEOR), and Translation Edit Rate (TER) scores, to provide a more complete picture of model performance.

Due to the hyperparameter search and ablation experiments being constrained by finite computational resources, not all combinations of model size and fine-tuning methods could be explored, and therefore, conclusions about scalability are extrapolated from a subset of possible configurations. Similarly, the ablation results are averaged over a limited number of random seeds, which may not fully characterize the variance in training outcomes. Additionally, the methodology focuses on the FLAN-T5 encoder-decoder family and does not include decoder-only architectures that are prevalent in many production LLM deployments. Although the methods presented here are compatible with decoder-only models with only minor code changes, empirical validation on such architectures remains an open direction.