Project

General

Profile

Actions

Feature #7467

closed

Establish Sequential Combination of MAV and MPV methods

Added by Vitaly Mordan over 8 years ago. Updated almost 5 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Tasks generation
Target version:
-
Start date:
06/28/2016
Due date:
06/28/2016
% Done:

0%

Estimated time:
Published in build:

Description

Global VTG strategy should present an infrastructure beyond current strategies, which is able to choose the needed strategy dynamically in optimal way. Global strategy will be default one for users, which do not know much about strategies. More advanced users could use it or the usual strategies.

The first Global strategy, which should become default for us, will unite strengths of MAV and MPV and will minimize the influence of their drawbacks. I suggest the following algorithm, which is based on the latest experiments (the first version):
1. Launch the first iteration of MAV for all rules. About 90% of cases will be solved in more optimized way (since MAV is more optimized). If some rule did not get a verdict Safe/Unsafe, then go to the next step.
2. If only a few rules did not get a verdict (due to heuristic time limit in MAV), launch MPV strategy Sep for them (for reuse of caches of verifier). If there was only one such rule, launch SBT-SA strategy for it. If MAV could not prove Safe for some rules, and most of them did not get any verdict, launch MPV strategy Relevance (as more powerful), but without those rules, which violated MAV internal limits – they should be checked separately by means of MPV-Sep/SBT-SA.

The suggested Global strategy has the following advantages:
1. optimizations of MAV for most of the tasks;
2. dynamic decomposition of MPV for complex tasks;
3. results of the first iteration of MAV will be reused;
4. much more powerful and flexible instrument in a contradiction to static decomposition (#7392).

In order to implement the Global strategy we need:
1. Add automata for all rules (at least from basic set) – this is almost done in #7448.
2. Add support of several verifiers to be able to switch on-the-fly between MAV and MPV (#7435).
3. Automatically change strategy in case of 1 rule (#7466).

This feature also can be viewed as a new method of dynamic specification decomposition.


Files

groups.ods (17.3 KB) groups.ods Vitaly Mordan, 08/17/2016 05:38 PM
job.json (1.42 KB) job.json Vitaly Mordan, 08/26/2016 03:10 PM
task-client.json (1.78 KB) task-client.json Vitaly Mordan, 08/26/2016 03:10 PM

Related issues 4 (0 open4 closed)

Related to Klever - Feature #7392: Support groupping of rule specifications to be checked togetherClosedEvgeny Novikov06/24/201606/24/2016

Actions
Blocked by Klever - Feature #7435: Support of several verifiers is requiredRejected07/29/2016

Actions
Blocked by Klever - Feature #7466: Automatically change multiple-strategies to separeted in case of 1 ruleRejectedVitaly Mordan08/12/2016

Actions
Follows Klever - Feature #7448: Add property automata for all rulesRejectedVitaly Mordan06/27/201606/27/2016

Actions
Actions #1

Updated by Evgeny Novikov over 8 years ago

  • Category changed from Program fragments generation to Tasks generation
  • Priority changed from High to Normal

This is actually yet another verification tasks generation strategy. I don't see any sense in some "globalization". Implementations of verification tasks generation strategies should allow to combine them in some way.

Actions #2

Updated by Vitaly Mordan over 8 years ago

The main idea of this strategy is that it can and should be used as default by most of the users, including our usual launches (except some experiments). Moreover it will be constructed on top of the current strategies (in VTG instead of strategy itself). At last, this "global" (or "default") strategy will be the only one after thorough evaluation.

Actions #3

Updated by Vitaly Mordan over 8 years ago

The complexity of each rule was evaluated with help of MPV strategy All.
Complexity represent the number of relevant modules.
This statistic shows:
1. the most complex rules are 39 and 43;
2. for some rules (parts of 147) EMG cannot generate any modules, for which they will be relevant;
3. overall only 1.46 rules (of 30) are relevant per one checked module;
4. maximal number of relevant rules per one task is 10 (for 2 verification tasks).
This statistics should be considered in design of this Global strategy.

Actions #4

Updated by Vitaly Mordan over 8 years ago

First version of Global strategy was implemented in branch new_format_of_strategies.
First experiments already show its major potential: overall speedup for the whole system is about 4 times (30 non-grouped rules and all modules), whereas losses are less than 0.1% of all verdicts.

Updated by Vitaly Mordan over 8 years ago

This strategy unified state-of-the art methods for verifying several rules at once as their Sequential Combination. Parameters were adjusted and can be already used by default. Sequential combination solves even more tasks, than separated strategies, but spends resources much more optimal.

Sequential Combination was implemented in branch new_format_of_strategies, which has number of fixes and improvements. Currently in that branch all our 30 basic rules are supported.
Currently there are the following limitations for Sequential Combination, which should be resolved in other issues if needed:
1. Does not support MEA (MPV does not support it - therefore in order to use MEA only MAV is applicable).
2. Does not support bug kinds (issue #7535).
3. Does not support VerifierCloud (as well as MAV for immediate printing of error traces).

Example of configuration for Sequential Combination is in attached files.

Actions #6

Updated by Vitaly Mordan almost 8 years ago

  • Subject changed from Establish Global VTG strategy to Establish Sequential Combination of MAV and MPV methods
Actions #7

Updated by Evgeny Novikov almost 5 years ago

  • Status changed from Resolved to Rejected
Actions

Also available in: Atom PDF