Postprocessing

Overview

The postprocessing functionality of AutoBench is dedicated to extracting benchmark KPIs from the output of benchmark job. Each benchmark is post-processed using predefined Perl templates. It also queries DCDB to extract performance counters and energy consumption, merges the data and represents them in CSV format.

Default Configuration

Templates are mapped to benchmarks via autobench/postprocessing/config.yaml, which determines the appropriate template for KPI extraction based on hardware components. For standardizing frequency values, reference autobench/postprocessing/frequency.yaml, as some components may use numeric codes (e.g., 0, 1, 2) for frequency settings.

To obtain the energy metric from your benchmark experiments, include the --powervalue option when using the postprocessing CLI command.

Setup

Postprocessing employs Perl, and the setup can be configured via Carton or within the local user directory.

Using Carton (Perl module dependency manager)

cd autobench/postprocessing
carton install

Local

#Set up local::lib
cpan local::lib

## in bash
eval $(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)

## cpanm
cpan App::cpanminus

##install
cpanm -L ~/home/test Module::Name

export PERL5LIB=~/home/test:$PERL5LIB

Custom templates

To define templates for KPI extraction, you can use autobench/postprocessing/templates/idle.tt as a base. In this template, START: [%start_time%] specifies a value to be captured after START:, naming it start_time. Outputs can also be simplified with [%ignore1%], which excludes everything between EXECUTING and END: [%end_time%]. You can define ignore variables up to [%ignore20%].

START: [%start_time%]
EXECUTING
[%ignore1%]
END: [%end_time%]
DONE
[%ignore2%]

Usage

To use the postprocessing simply use:

autobench postprocessing --start

Debugging

Creating a time template may not work perfectly with custom templates right away. It is advisable to construct the template step-by-step, incorporating variables one at a time.