Machine Learning Hyper-parameter Optimisation using Particle Swarms

Description

The accuracy and generalisability of a machine learning model is determined by the right choice of hyper-parameters.  All models have hyper-parameters (even non-parametric models) and the wrong choice will result in a poor fit to the data, and useless predictions.  Finding the optimal set of hyper-parameter for any model is always the most time consuming part of machine learning, and is usually done by extensive (and sometimes exhaustive) grid searches, or highly variable random selection.  An alternative is to use a particle swarm optimizaiton (PSO) algorithm which is a computational method that uses a population of candidate solutions that are moved around the search-space to iteratively improves a solution toward a given measure of quality. In this project you will implement a particle swarm algorithm for hyper-parameter optimization of machine learning models in python, test the performance of different numerical conditions, and compare the result (accuracy and computational efficiency) to random and grid searching methods. Data sets will be provided.

Goals

To produce a python module for a library for general use by machine learning researchers, and a scientific publication

Requirements

Python programming and experience in data science, machine learning and software engineering is essential (such as COMP3720, COMP4660, COMP4670, COMP6670, COMP8420).  Familiarity with scikit-learn is desirable.

Gain

This is a 24cp project.

Keywords

software engineering, python, machine learning, optimization, evolutionary computing

Updated:  10 August 2021/Responsible Officer:  Dean, CECS/Page Contact:  CECS Marketing