Output CSV
Graph are automatically generated for all tested variables combinations.
On top of graph NPF has two output modules to extract the generated data. Recall NPF also uses an internal database to remember every values generated by the test script.
Single CSV file
This is created with --single-output filename.csv
The following is the output of npf-compare.py --test integration/math.npf --single-output test.csv, a test script that prints the input vairable “N”, its logarithm and square product for N=[1-32] :
index |
build |
test_index |
N |
N |
LOG |
EXP |
run_index |
0 |
Local |
0 |
1 |
1.0 |
0.0 |
2.0 |
0 |
1 |
Local |
0 |
1 |
1.0 |
0.0 |
2.0 |
1 |
2 |
Local |
0 |
1 |
1.0 |
0.0 |
2.0 |
2 |
[…] |
|||||||
93 |
Local |
31 |
32 |
32.0 |
3.47 |
4294967296.0 |
0 |
94 |
Local |
31 |
32 |
32.0 |
3.47 |
4294967296.0 |
1 |
95 |
Local |
31 |
32 |
32.0 |
3.47 |
4294967296.0 |
2 |
The columns are:
index: An index for the line build: The name of the build/repository, “Local” is the default name for no repository. If you used npf-compare “local:Serie” the name woult be Serie test_index: For each build, the index of the combination of variable. N: This is the input variable “N” N: This is the output result-type “N” LOG: This is the output result type “LOG” EXP: This is the output result type “EXP” run_index: For each test_index, the index of the run. Goes from 0 to
--config n_runs=X
Example to open the file in python and compute the mean for each variable:
import pandas as pd
df = pd.read_csv('test.csv')
df.groupby('test_index').mean()
index N N.1 LOG EXP run_index
test_index
0 1.0 1.0 1.0 0.00 2.000000e+00 1.0
1 4.0 2.0 2.0 0.69 4.000000e+00 1.0
2 7.0 3.0 3.0 1.10 8.000000e+00 1.0
[...]
29 88.0 30.0 30.0 3.40 1.073742e+09 1.0
30 91.0 31.0 31.0 3.43 2.147484e+09 1.0
31 94.0 32.0 32.0 3.47 4.294967e+09 1.0
While this single output file contains all the data and allow complete manipulation, it forces the user to do aggregate by themselves.
Multiple CSV files
The –output lines will create one CSV per output type, just before generating the graphs. Therefore series, graph transformations, … all apply before the CSV is generated.
With –output-columns the list of columns can be selected among:
mean/average: The mean value of all runs
min: The min value of all runs
max: The max values of all runs
perc[0-9]+: The configurable percentile for all runs
med/median/perc50: The median value for all runs
std: The standard deviation
nres/n: The number of runs
first: The first value of the runs
last: The last value of the runs
all: One column per value of the run
The following is the output of npf-compare.py --test integration/math.npf --output test.csv --output-columns x mean median, using the same test script as above :
1 0.0 0.0 |
2 0.69 0.69 |
3 1.1 1.1 |
[…] |
30 3.4 3.4 |
31 3.43 3.43 |
32 3.47 3.47 |
There is no variance in this test, so the mean is equal to the median.