Speaker: Szu Hui NG, National University of Singapore (NUS)

Inviter: 胡庆培

Title: Trajectory-Based Multi-Objective Hyperparameter Optimization for Model Retraining

Language: Chinese

Time & Venue: 2024.06.03 10:30-11:30 南楼N208

Abstract: Training machine learning models inherently involves a resource-intensive and noisy iterative learning procedure that allows one to observe the model performance over epochs or time intervals. In multi-objective hyperparameter optimization scenarios however, the insights gained from the iterative learning procedure typically remain under-utilized. Specifically, we notice that tracking the model performance across multiple epochs under a hyperparameter setting creates a trajectory in the objective space, but the trade-offs along the trajectories are often overlooked, despite their potential to offer valuable insights to decision-making for model retraining. In this work, we propose to enhance the multi-objective hyperparameter optimization problem by having training epochs as an additional decision variable to incorporate trajectory information. Correspondingly, we present a novel trajectory-based multi-objective Bayesian optimization algorithm characterized by two features: 1) an acquisition function that captures the improvement made by the predictive trajectory of any hyperparameter setting and 2) a multi-objective early stopping mechanism that determines when to terminate the trajectory to maximize epoch efficiency, to solve this enhanced problem.