High Order Optimization Methods In Training Deep Learning Models
Optimization algorithm is the backbone for training machine learning (ML) models. Besides of widespread first-order algorithms, high order algorithms emerge as an alternative. We demystify some commonly misunderstandings that discourage their adoption in ML practitioner community. In most of the cases, high order algorithms are neither too complex nor too expensive. Two high order algorithms will be described briefly. We present some numerical experiments to illustrate the perfor- mance of AdaHessian as a high order algorithm representative.
Research paper: High Order Optimization Methods In Training Deep Learning Models
Tran Ngoc Nguyen & Luu Quang Nhien
3/21/2025
NEW
To address the challenges of providing quick and plausible explanations in Explainable AI (XAI) for object detection models, we introduce the Gaussian Class Activation Mapping Explainer (G-CAME).
NEW
The rising use of Artificial Intelligence (AI) in human detection on Edge camera systems has led to accurate but complex models, challenging to interpret and debug.
NEW
In recent years, reinforcement learning (RL) has emerged as a new, promising way to solve old problems. The algorithms’ role in finding approximate solutions in NP-hard complexity became crucial for developing modern intelligent decisions.
QaiDora Products
Trusted by
Contact us
