Papers Explained Review 13: Model Merging | by Ritvik Rastogi | Apr, 2025

Mannequin merging strategies supply a robust option to mix a number of fine-tuned fashions, leveraging their strengths to boost efficiency with out extra coaching. This text explores varied mannequin merging methods and offers pattern configurations utilizing MergeKit, demonstrating methods to apply these strategies in real-world situations. Whether or not you’re optimizing mannequin ensembles or exploring weight-space geometry, this information will assist you to navigate the panorama of mannequin merging successfully.

1. Model Soup
2. Spherical Linear Interpolation (SLERP)
3. Nearswap
4. Task Arithmetic
5. Trim, Elect Sign & Merge (TIES)
6. Drop And REscale (DARE)
7. Model Breadcrumbs
8. Model Stock
9. NuSLERP (Normalized SLERP)
10. Drop and rEscaLe via sampLing with mAgnitude (DELLA)
11. Select, Calculate, and Erase (SCE)

Mannequin Soup refers back to the easy concept of averaging mannequin weights throughout a number of superb‑tuned fashions. The underlying assumption is that fashions superb‑tuned from the identical pre-trained spine (and on associated duties or domains) lie in a “linked” area of parameter house in order that their easy linear mixture can yield improved generalization.

Given a set of fashions with weights (W_1, W_2,…, W_N) and nonnegative coefficients (α_1, α_2, … α_N) that sum to 1, the merged mannequin is:

Mannequin Soups: Averaging Weights of A number of Tremendous-Tuned Fashions Improves Accuracy With out Retraining 2203.05482.

Parameters

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.

fashions:
- mannequin: meta-llama/Llama-3.1-8B-Instruct
parameters:
weight: 0.5
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.15
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.35
merge_method: linear
dtype: float16

Back To Top

SLERP performs interpolation alongside a terrific circle on the sphere of normalized weight vectors. Somewhat than a straight (Euclidean) interpolation, it preserves angular relationships. That is particularly helpful when weight vectors are normalized, making certain that the interpolated mannequin stays “on the manifold.”

For 2 weight vectors (a) and (b) and an interpolation parameter (t in [0,1]):

Parameters

• t (Interpolation Issue): Controls the place alongside the good circle between the 2 fashions.

fashions:
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
merge_method: slerp
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
t: 0.5
dtype: float16

Back To Top

“Nearswap” is designed to determine and leverage areas within the parameter house the place two fashions are “shut” (i.e. related) whereas merging. In apply, the tactic partitions the mannequin’s parameters (or layers) after which “swaps” or averages solely these parameters whose distinction is inside a specified threshold.

1. Compute the space:

2. Merge based mostly on the edge τ:

Parameters

• t (Similarity Threshold ()): Distance beneath which parameters are thought of “close to” and thus eligible for swapping.

fashions:
- mannequin: meta-llama/Llama-3.1-8B-Instruct
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
merge_method: nearswap
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
t: 0.5
dtype: float16

Back To Top

Activity Arithmetic leverages the concept mannequin parameters usually encode “instructions” associated to particular duties. By subtracting the widespread (shared) illustration and including a task-specific element, one can compose fashions that higher carry out a composite job.

Modifying Fashions with Activity Arithmetic 2212.04089.

Parameters

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: task_arithmetic
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
dtype: float16

Back To Top

The TIES-MERGING algorithm addresses interference points when merging a number of task-specific fashions by using a three-step course of: Trim, Elect Signal, and Disjoint Merge. This course of goals to create a merged mannequin that successfully combines the information from particular person task-specific fashions whereas mitigating conflicting parameter updates.

1. For every job vector, retain the highest ok% of parameters with the best magnitudes and set the remaining (backside (100 — ok)%) to zero. This creates a trimmed job vector.
2. For every parameter, calculate the entire magnitude of constructive and damaging indicators throughout all trimmed job vectors. Assign the signal with the bigger whole magnitude to the merged mannequin’s signal vector.
3. For every parameter, outline a set containing job indices the place the signal of the trimmed job vector agrees with the elected signal. Compute the disjoint imply by averaging the values of the parameter.

TIES-Merging: Resolving Interference When Merging Fashions 2306.01708.

Parameters

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors
• density (ok) — fraction of weights in variations from the bottom mannequin to retain

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: ties
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.7
dtype: float16

Back To Top

The DARE (Drop and Rescale) algorithm reduces redundancy in delta parameters (modifications from pre-training to fine-tuning) of huge language fashions. It randomly units a proportion of delta parameters to zero and rescales the remaining ones by an element of 1/(1-p), the place p is the drop price, then provides them again to the pre-trained parameters.

1. Given a pre-trained LM with weights W_PRE and a fine-tuned LM for job t with weights W_SFT_t, the delta parameters (Δ_t) are computed.
2. Randomly set a proportion p of the delta parameters to zero utilizing a Bernoulli distribution. For every factor in Δ_t, a random variable m_t is drawn from Bernoulli(p).
3. The remaining non-zero delta parameters are rescaled by an element of 1 / (1 — p) to compensate for the dropped values
4. Lastly, the rescaled delta parameters (Δ̂_t) are added again to the pre-trained weights W_PRE to acquire the DARE-adapted weights W_DARE_t

DARE can be utilized both with the signal consensus algorithm of TIES (dare_ties) or with out (dare_linear).

Language Fashions are Tremendous Mario: Absorbing Skills from Homologous Fashions as a Free Lunch 2311.03099.

Parameters (dare_ties)

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors
• Density (ok) — fraction of weights in variations from the bottom mannequin to retain

Parameters (dare_linear)

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: dare_ties
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.7
dtype: float16

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: dare_linear
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
dtype: float16

Back To Top

An extension of job arithmetic that discards each small and intensely massive variations from the bottom mannequin. The Mannequin Breadcrumbs algorithm can be utilized with (breadcrumbs_ties) or with out (breadcrumbs) the signal consensus algorithm of TIES.

1. Activity Vector Creation: For every fine-tuned mannequin comparable to a particular job, calculate the distinction between its weights and the unique pre-trained basis mannequin’s weights. This distinction vector is known as the duty vector.
2. Outlier and Negligible Perturbation Elimination: Outline two thresholds, β (left tail) and γ (proper tail), representing percentages. Masks out (set to zero) the weights within the backside β% and the highest (100-γ)% of the sorted weights in every layer. This eliminates each massive outliers and negligible perturbations.
3. Combining Activity Vectors: Mixture the masked job vectors throughout all duties by summing them.
4. Scaling and Integration: Scale the summed job vectors by a energy parameter (α) and add them to the unique pre-trained mannequin’s weights.

Mannequin Breadcrumbs: Scaling Multi-Activity Mannequin Merging with Sparse Masks 2312.06795.

Parameters:

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors
• density — fraction of weights in variations from the bottom mannequin to retain
• gamma — fraction of largest magnitude variations to take away

Word that gamma corresponds with the parameter β described within the paper, whereas density is the ultimate density of the sparsified tensors (associated to γ and β by density = 1 — γ — β).

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: breadcrumbs
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.9
gamma: 0.01
dtype: float16

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: breadcrumbs_ties
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.9
gamma: 0.01
dtype: float16

Back To Top

The Mannequin Inventory algorithm is a cost-efficient weight merging methodology that goals to enhance mannequin efficiency by approximating the middle of weight distribution (µ) utilizing a pre-trained mannequin as an anchor level and some fine-tuned fashions. It leverages the geometric properties of weight vectors, particularly the angle between them, to find out the optimum merging ratio.

• Airplane Definition: A airplane is outlined utilizing the pre-trained mannequin’s weight vector (w0) and two fine-tuned fashions’ weight vectors (w1 and w2). This airplane represents the search house for the merged weight.
• Perpendicular Foot Calculation: The algorithm goals to search out the purpose on this airplane (wH) that’s closest to the middle of the burden distribution (µ). This level is the perpendicular foot from µ to the airplane.

θ is the angle between the 2 fine-tuned mannequin weight vectors (w1 and w2).

wH is the merged weight vector.

w0 is the pre-trained mannequin’s weight vector.

(w1 + w2)/2 represents the typical of the 2 fine-tuned weight vectors which pertains to w12 within the authentic textual content.

• Interpolation Ratio: The interpolation ratio t = 2 * cos(θ) / (1 + cos(θ)) determines the contribution of the averaged fine-tuned weights and the pre-trained weight to the merged weight. This ratio is solely depending on the angle θ. A smaller angle means much less reliance on the pre-trained mannequin.
• Extension to N Tremendous-tuned Fashions:

t = N * cos(θ) / (1 + (N — 1) * cos(θ))

θ is the angle between the pre-trained mannequin and the N fine-tuned fashions.

w(N)H is the merged weight vector.

Mannequin Inventory: All we want is only a few fine-tuned fashions 2403.19522.

Parameters:

• filter_wise: if true, weight calculation might be per-row reasonably than per-tensor. Not beneficial.

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
merge_method: model_stock
base_model: meta-llama/Llama-3.1-8B-Instruct
dtype: float16

Back To Top

NuSLERP modifies customary SLERP by explicitly normalizing the burden vectors earlier than interpolation. This “normalized” model is especially helpful when fashions have been educated with totally different scaling (e.g. as a consequence of adaptive normalization layers) in order that the interpolation doesn’t “combine” incompatible scales.

Parameters:

• weight: relative weighting of a given tensor
• nuslerp_flatten: set to false to do row-wise/column-wise interpolation as an alternative of treating tensors as vectors
• nuslerp_row_wise: SLERP row vectors as an alternative of column vectors

fashions:
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.5
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.5
merge_method: nuslerp
base_model: meta-llama/Llama-3.1-8B-Instruct
dtype: float16

Back To Top

DELLA can be utilized with (della) or with out (della_linear) the signal elect step of TIES.

1. Drop: This step makes use of a novel magnitude-based pruning strategy known as MAGPRUNE:

• Rank delta parameters for every node within the neural community based mostly on their magnitude (absolute worth).
• Assign a drop chance (Pd) to every parameter inversely proportional to its magnitude. Bigger magnitude parameters have a decrease chance of being dropped. That is managed by a hyperparameter ∆ that determines the step measurement between possibilities.
• A hyperparameter ‘p’ controls the typical drop chance. ‘ϵ’ influences the minimal drop chance (pmin = p — ϵ/2).
• Stochastically drop delta parameters based mostly on their assigned possibilities. A parameter is about to zero if dropped.
• Scaling: Rescale the remaining (undropped) delta parameters by 1 / (1 — pi) the place pi is the drop chance of the i-th parameter. This compensates for the impact of dropping parameters and ensures the mannequin’s output embeddings are preserved.

2. Elect: Decide the dominant path for every parameter place by calculating the signal of the sum of all corresponding delta parameters throughout consultants. Choose (elect) solely the delta parameters at place which have the identical signal because the dominant path.

3. Fuse: Calculate the typical of the elected delta parameters for every place.

4. Receive Merged Mannequin: Add the fused delta parameters (scaled by an element λ) to the bottom mannequin’s parameters.

DELLA-Merging: Lowering Interference in Mannequin Merging by way of Magnitude-Primarily based Sampling 2406.11617.

Parameters:

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda — scaling issue utilized after weighted sum of job vectors
• density — fraction of weights in variations from the bottom mannequin to retain
• epsilon — most change in drop chance based mostly on magnitude. Drop possibilities assigned will vary from density — epsilon to density + epsilon. (When deciding on values for density and epsilon, be certain that the vary of possibilities falls inside 0 to 1)

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: della
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.7
epsilon: 0.01
dtype: float16

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: della_linear
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
density: 0.7
epsilon: 0.01
dtype: float16

Back To Top

The SCE (Choose, Calculate, Erase) methodology is a method for merging a number of goal LLMs that share the identical structure and scale however have been individually fine-tuned with information from totally different supply LLMs. It operates on “fusion vectors,” which symbolize the distinction in weights between a pivot LLM and every goal LLM after the pairwise information fusion stage.

1. For every parameter matrix within the set of fusion vectors, choose the highest ok% of components with the best variance throughout the totally different goal LLMs.
2. For every parameter matrix, calculate the merging coefficient for every goal LLM because the sum of squares of the chosen components in its corresponding filtered fusion vector, normalized by the entire sum of squares throughout all goal LLMs for that matrix.
3. For every parameter within the filtered fusion vectors, sum the values throughout all goal LLMs. If the sum for a given parameter is constructive (or damaging), set all damaging (or constructive) values for that parameter to zero. This eliminates conflicting replace instructions.
4. After the SCE course of, the ultimate merged LLM’s parameter matrix is calculated as Activity Arithmetic:

FuseChat: Data Fusion of Chat Fashions 2408.07990.

Parameters:

• weight (α) — relative (or absolute if normalize=False) weighting of a given tensor
• normalize — if true, the weights of all fashions contributing to a tensor might be normalized. Default habits.
• lambda— scaling issue utilized after weighted sum of job vectors
• select_topk — fraction of components with the best variance within the delta parameters to retain.

fashions:
- mannequin: NousResearch/Hermes-3-Llama-3.1-8B
parameters:
weight: 0.3
- mannequin: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
parameters:
weight: 0.7
merge_method: sce
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
lambda: 0.5
select_topk: 0.7
dtype: float16

Back To Top

• Mannequin Soups: Averaging Weights of A number of Tremendous-Tuned Fashions Improves Accuracy With out Retraining 2203.05482
• Modifying Fashions with Activity Arithmetic 2212.04089
• TIES-Merging: Resolving Interference When Merging Fashions 2306.01708
• Language Fashions are Tremendous Mario: Absorbing Skills from Homologous Fashions as a Free Lunch 2311.03099
• Mannequin Breadcrumbs: Scaling Multi-Activity Mannequin Merging with Sparse Masks 2312.06795
• Mannequin Inventory: All we want is only a few fine-tuned fashions 2403.19522
• DELLA-Merging: Lowering Interference in Mannequin Merging by way of Magnitude-Primarily based Sampling 2406.11617
• FuseChat: Data Fusion of Chat Fashions 2408.07990