Some Model Merging Insights

Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive computation. My recent research focuses on: 1. Comparing the effects and effectiveness of data mixing training and model merging. 2. Analyzing the impact of parameters at layer-wise level on LLMs' performance, and whether better model merging can be achieved at the layer-wise level.