Feature #1000: Configure like tests for changes in kernel core API - Linux Driver Verification - Open-Source Projects

Actions

Copy link

Feature #1000

open

Configure like tests for changes in kernel core API

Added by Alexey Khoroshilov over 13 years ago. Updated over 9 years ago.

Status:

Open

Priority:

High

Assignee:

Evgeny Novikov

Category:

Tests and QA

Start date:

03/30/2011

Due date:

% Done:

Estimated time:

Published in build:

Description

We should have a configure like tests that quickly checks compatibility of rules with the given version of Linux kernel.

The tests can be based on unit tests for our rules complimented by a framework to check compatibility of aspect definitions (rules) with a kernel+config (given as a folder in a file system, first of all).

Actions

Copy link

Updated by Pavel Shved about 13 years ago

Category set to Tests and QA

As we discussed on Planning 15,

We have a script source:ldv-tests/regr-tests/ldv-test that runs a test set against a current toolset version (we used it for benchmarking different verifiers); it uses the good old regr-test.pl as a backend. If we could supply different kernel (not the one that's hardcoded in the tests), we could just run the unit-tests for models against the latest kernel, and check if the results match. This could serve as such test, and we don't have to invest much in that.

Actions

Copy link

Updated by Pavel Shved almost 13 years ago

Priority changed from High to Normal

Actions

Copy link

Updated by Evgeny Novikov almost 12 years ago

Priority changed from Normal to High

IMHO, this feature request is becoming more and more necessary. A recent trouble due to it was that 08_1a model didn't contain bindings with new kernels that have changed the interface. Although this model has passed its model tests this didn't guarantee that this was a good model.
I guess that a person who will deal this feature request also should completely redesign the current approach of regression testing, that includes:

Much more flexibility in regression task setting (ideal verdicts, configuring on fly, different launch settings, etc.).
Ability to specify how to interpret results and compare them with the original task (e.g. support so-called unstable drivers).
Fixing existing test sets (may be filtering useless tests, standardization of tests development, etc.).
Fixing components that depend on regression testing like QA infrastructure.

Rather good work for diploma or term paper.

Actions

Copy link

Updated by Alexey Khoroshilov almost 12 years ago

One more idea is to implement support for using git-based kernels in our test infrastructure.
The main purpose is to run verification of known bugs from mainline drivers.

Actions

Copy link