Feature #1000
open
Configure like tests for changes in kernel core API
Added by Alexey Khoroshilov over 13 years ago.
Updated over 9 years ago.
Description
We should have a configure like tests that quickly checks compatibility of rules with the given version of Linux kernel.
The tests can be based on unit tests for our rules complimented by a framework to check compatibility of aspect definitions (rules) with a kernel+config (given as a folder in a file system, first of all).
- Category set to Tests and QA
As we discussed on Planning 15,
We have a script source:ldv-tests/regr-tests/ldv-test that runs a test set against a current toolset version (we used it for benchmarking different verifiers); it uses the good old regr-test.pl as a backend. If we could supply different kernel (not the one that's hardcoded in the tests), we could just run the unit-tests for models against the latest kernel, and check if the results match. This could serve as such test, and we don't have to invest much in that.
- Priority changed from High to Normal
- Priority changed from Normal to High
IMHO, this feature request is becoming more and more necessary. A recent trouble due to it was that 08_1a model didn't contain bindings with new kernels that have changed the interface. Although this model has passed its model tests this didn't guarantee that this was a good model.
I guess that a person who will deal this feature request also should completely redesign the current approach of regression testing, that includes:
- Much more flexibility in regression task setting (ideal verdicts, configuring on fly, different launch settings, etc.).
- Ability to specify how to interpret results and compare them with the original task (e.g. support so-called unstable drivers).
- Fixing existing test sets (may be filtering useless tests, standardization of tests development, etc.).
- Fixing components that depend on regression testing like QA infrastructure.
Rather good work for diploma or term paper.
One more idea is to implement support for using git-based kernels in our test infrastructure.
The main purpose is to run verification of known bugs from mainline drivers.
- Assignee set to Ilya Shchepetkov
Decided to be implemented by Ilya Shchepetkov.
- Status changed from New to Open
- Assignee changed from Ilya Shchepetkov to Evgeny Novikov
Also available in: Atom
PDF