Repositorio UVigo

KPIs-based clustering and visualization of HPC jobs: a feature reduction approach

Investigo Repository

KPIs-based clustering and visualization of HPC jobs: a feature reduction approach

Halawa, Mohamed Soliman; Díaz Redondo, Rebeca Pilar; Fernández Vilas, Ana
DATE : 2021-02
UNESCO SUBJECT : 1209.03 Análisis de Datos ; 1203.17 Informática


High-Performance Computing (HPC) systems need to be constantly monitored to ensure their stability. The monitoring systems collect a tremendous amount of data about different parameters or Key Performance Indicators (KPIs), such as resource usage, IO waiting time, etc. A proper analysis of this data, usually stored as time series, can provide insight in choosing the right management strategies as well as the early detection of issues. In this paper, we introduce a methodology to cluster HPC jobs according to their KPI indicators. Our approach reduces the inherent high dimensionality of the collected data by applying two techniques to the time series: literature-based and variance-based feature extraction. We also define a procedure to visualize the obtained clusters by combining the two previous approaches and the Principal Component Analysis (PCA). Finally, we have validated our contributions on a real data set to conclude that those KPIs related to CPU usage provide the best cohesion and separation for clustering analysis and the good results of our visualization methodology.

Show full item record

Files in this item

Attribution 4.0 International Except where otherwise noted, this item's license is described as Attribution 4.0 International
2013 Universidade de Vigo, Todos los derechos reservados
Calidad So9001