Bunsen is a test result storage and analysis toolkit
https://sourceware.org/bunsen/  

This is Free Software, licensed under the GNU Lesser General Public
License v3.0 or later.  See lgpl+gpl-3.0.txt for details.


Quick HOWTO on playing with this system:

-------------------- get analysis software

"build" and install the software

% configure --prefix=$SOMEWHERE   # checks for python3 prerequites: git (AKA GitPython), argparse, sqlite3,
                                  #    logging, natsort, jinja2, flask, tqdm, typing_extensions, jq, lynx
                                  # optional: torch
% make
% make check                      # some more prequisites: jq, diff
% make install
% PATH=$SOMEWHERE/bin:$PATH
or else: just run it out of source tree:
% PATH=$sourcetree/bin:$PATH

-------------------- get analysis software via docker/podman

% podman pull quay.io/bunsen-testsuite-analysis/bunsen
% podman run --rm -i -t quay.io/bunsen-testsuite-analysis/bunsen /bin/sh
  % ls /home/bunsen
  % ls /usr/local/bin  # runnable toolkit

-------------------- get analysis software as a Fedora RPM

configure following DNF repository: https://copr.fedorainfracloud.org/coprs/mcermak/bunsen/
and install the bunsen.rpm package

-------------------- automated copr builds

1) install the "tito" tool: e.g. https://koji.fedoraproject.org/koji/packageinfo?packageID=9382
2) Go to the bunsen git repo and tag the release: `tito tag`.  This will also bump the package NVR.
3) `git push --follow-tags origin`
4) Trigger the rebuild.
5) Soon fresh build should show up: https://copr.fedorainfracloud.org/coprs/mcermak/bunsen/
6) FTR: https://docs.fedoraproject.org/en-US/quick-docs/publish-rpm-on-copr/
7) Have fun!

8) To create a testing RPM build locally, `tito build  --srpm --test` can be used to create
   a SRPM based on current git branch head.  Testing uncommited changes is a tito wip:
   https://github.com/rpm-software-management/tito/issues/441

-------------------- get some test result logs

Create git repo to contain the testrun log files

% gitrepo=SOMEPLACE
% mkdir $gitrepo
% git init $gitrepo

Import any dejagnu-type log files into a new commit in the testrun repo

% t-upload-git-push $gitrepo any-unique-git-tag-$$ */*.log */*.sum

Pull some content from a remote testrun repo by pattern matching some tags

% cd $gitrepo
% git fetch git://example.com/ 'refs/tags/wild*:refs/tags/wild*'

Pull all content from a remote testrun repo

% cd $gitrepo
% git remote add r git://example.com/
% git pull --tags

-------------------- run analysis

Run the analysis pipeline to refresh database

% cd $gitrepo
% pipeline # add --loglevel=debug for verbosity

Any time, inspect the resulting database

% cd $gitrepo
% sqlite3 bunsen.sqlite3
% sqlitebrowser bunsen.sqlite3

Look at the basic testrun data in git

% cd $gitrepo
% r-show-testrun any-unique-git-tag-$$
% r-show-testrun any-unique-git-tag-$$ path/to/file.log

Compare testruns

% cd $gitrepo
% r-diff-testruns $tag1 $tag2 $tag3 ...
% r-diff-testruns --expfile-like '%substring%' $tag1 $tag2  # filtering 

Show test results from --baseline to --latest as a grid view

% cd $gitrepo
% R-show-testcases --source-git=$project-source-checkout --baseline=$refspec --latest=$refspec >output.html
% R-show-testcases --source-git=$project-source-checkout --baseline=$refspec --latest=$refspec --has-expfile-like '%substring%' >output.html
% R-show-testcases --source-git=$project-source-checkout --baseline=$refspec --latest=$refspec --slice-expnames 100 >output.html

(Here, $project-source-checkout is a local Git checkout of the project source code,
used for scanning the project's commit history; if the script uses too much RAM,
--slice-expnames limits the number of expfiles and memory usage at the cost of
taking more time.)

Browse the database with a web browser

% firefox http://localhost:8003/ &
% r-httpd-browse 

-------------------- a complete working example:

cd /tmp
git clone git://sourceware.org/git/bunsen.git bunsen
cd bunsen
./configure --prefix=`pwd`/INST; make; make install
PATH=`pwd`/INST/bin:$PATH

cd /tmp
git init bunsendb
cd bunsendb

git fetch git://sourceware.org/git/bunsendb.git 'refs/tags/buildbot/binutils-gdb*:refs/tags/buildbot/binutils-gdb*'

pipeline

tag=`git tag | grep buildbot| shuf| head -1`
r-show-testrun $tag

tag2=`git tag | grep buildbot| shuf| head -1`
r-diff-testruns --dgexpfile 'gdb.base/break.exp' $tag $tag2

r-dejagnu-testcase-classify --mode=train `git tag | shuf | head -20`
r-dejagnu-testcase-classify $tag

-------------------- quick local import & comparison of test directories

t-quick-process-testdir TREE1 TREE2 [TREE3 ...]
  # creates temporary local bunsendb git repo, sql index, runs pipeline, starts temporary r-httpd-browse locally
firefox http://localhost:PORT