Feature #7467
closedEstablish Sequential Combination of MAV and MPV methods
0%
Description
Global VTG strategy should present an infrastructure beyond current strategies, which is able to choose the needed strategy dynamically in optimal way. Global strategy will be default one for users, which do not know much about strategies. More advanced users could use it or the usual strategies.
The first Global strategy, which should become default for us, will unite strengths of MAV and MPV and will minimize the influence of their drawbacks. I suggest the following algorithm, which is based on the latest experiments (the first version):
1. Launch the first iteration of MAV for all rules. About 90% of cases will be solved in more optimized way (since MAV is more optimized). If some rule did not get a verdict Safe/Unsafe, then go to the next step.
2. If only a few rules did not get a verdict (due to heuristic time limit in MAV), launch MPV strategy Sep for them (for reuse of caches of verifier). If there was only one such rule, launch SBT-SA strategy for it. If MAV could not prove Safe for some rules, and most of them did not get any verdict, launch MPV strategy Relevance (as more powerful), but without those rules, which violated MAV internal limits – they should be checked separately by means of MPV-Sep/SBT-SA.
The suggested Global strategy has the following advantages:
1. optimizations of MAV for most of the tasks;
2. dynamic decomposition of MPV for complex tasks;
3. results of the first iteration of MAV will be reused;
4. much more powerful and flexible instrument in a contradiction to static decomposition (#7392).
In order to implement the Global strategy we need:
1. Add automata for all rules (at least from basic set) – this is almost done in #7448.
2. Add support of several verifiers to be able to switch on-the-fly between MAV and MPV (#7435).
3. Automatically change strategy in case of 1 rule (#7466).
This feature also can be viewed as a new method of dynamic specification decomposition.
Files
Updated by Evgeny Novikov over 8 years ago
- Category changed from Program fragments generation to Tasks generation
- Priority changed from High to Normal
This is actually yet another verification tasks generation strategy. I don't see any sense in some "globalization". Implementations of verification tasks generation strategies should allow to combine them in some way.
Updated by Vitaly Mordan over 8 years ago
The main idea of this strategy is that it can and should be used as default by most of the users, including our usual launches (except some experiments). Moreover it will be constructed on top of the current strategies (in VTG instead of strategy itself). At last, this "global" (or "default") strategy will be the only one after thorough evaluation.
Updated by Vitaly Mordan over 8 years ago
- File groups.ods groups.ods added
The complexity of each rule was evaluated with help of MPV strategy All.
Complexity represent the number of relevant modules.
This statistic shows:
1. the most complex rules are 39 and 43;
2. for some rules (parts of 147) EMG cannot generate any modules, for which they will be relevant;
3. overall only 1.46 rules (of 30) are relevant per one checked module;
4. maximal number of relevant rules per one task is 10 (for 2 verification tasks).
This statistics should be considered in design of this Global strategy.
Updated by Vitaly Mordan over 8 years ago
First version of Global strategy was implemented in branch new_format_of_strategies.
First experiments already show its major potential: overall speedup for the whole system is about 4 times (30 non-grouped rules and all modules), whereas losses are less than 0.1% of all verdicts.
Updated by Vitaly Mordan about 8 years ago
- File job.json job.json added
- File task-client.json task-client.json added
- Status changed from New to Resolved
This strategy unified state-of-the art methods for verifying several rules at once as their Sequential Combination. Parameters were adjusted and can be already used by default. Sequential combination solves even more tasks, than separated strategies, but spends resources much more optimal.
Sequential Combination was implemented in branch new_format_of_strategies, which has number of fixes and improvements. Currently in that branch all our 30 basic rules are supported.
Currently there are the following limitations for Sequential Combination, which should be resolved in other issues if needed:
1. Does not support MEA (MPV does not support it - therefore in order to use MEA only MAV is applicable).
2. Does not support bug kinds (issue #7535).
3. Does not support VerifierCloud (as well as MAV for immediate printing of error traces).
Example of configuration for Sequential Combination is in attached files.
Updated by Vitaly Mordan almost 8 years ago
- Subject changed from Establish Global VTG strategy to Establish Sequential Combination of MAV and MPV methods
Updated by Evgeny Novikov over 4 years ago
- Status changed from Resolved to Rejected