Project

General

Profile

Feature #8171

Get rid of repeated unknowns during multimodule verification

Added by Alexey Polushkin almost 2 years ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Program fragments generation
Target version:
Start date:
11/13/2017
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Published in build:

Description

Since, the result of verification of a module may be unknown, the result of verification each group, that contains this module also will be unknown. It's necessary to get rid of repeated unknowns that refers to the same reason of verdict.


Subtasks

Feature #8569: Increase length for problem descriptions up to 20 symbolsClosedIlja Zakharov

Feature #8570: Introduce data attributesClosedVladimir Gratinskiy

Feature #8571: Describe content of verification tasks generated by a multimodule strategy using data attributesClosedIlja Zakharov


Related issues

Related to Klever - Feature #8509: Introduce common prefixes for problems to group them togetherNew2017-10-19

Blocked by Klever - Feature #7167: Support for multiline patterns of unknown marksClosed2016-04-292017-10-25

Blocked by Klever - Feature #8425: Add support for unknown patterns without regular expressionsClosed2017-09-122017-10-25

Blocked by Klever - Feature #8495: Test regular expressions associationClosed2017-10-112017-10-25

History

#1 Updated by Evgeny Novikov almost 2 years ago

Why is this necessary? For me it seems very good to see the total number of problems as it is shown now. For instance, let's assume that one "bad" module breaks verification of 5 modules since it is added to corresponding verification objects, and there is another "bad" module that breaks verification of the only module. It would be nice to see all 5 unknowns in the first case to understand better than a corresponding issue should be likely fixed first.

#2 Updated by Ilja Zakharov almost 2 years ago

  • Priority changed from Low to High

I agree that the total number of unknowns is of interest. But at analyzing results of multimodule verification and comparing them with existing ones there is a difficulty of distinguishing actual problems of mutlimodule verification from problems caused by an insertion of a bad module in different verification objects. Currently, I have no good proposal how to deal with it.

#3 Updated by Evgeny Novikov almost 2 years ago

Ilja Zakharov wrote:

I agree that the total number of unknowns is of interest. But at analyzing results of multimodule verification and comparing them with existing ones there is a difficulty of distinguishing actual problems of mutlimodule verification from problems caused by an insertion of a bad module in different verification objects. Currently, I have no good proposal how to deal with it.

In my opinion, insertion of a "bad" module is exactly a problem of multimodule verification. And the only thing to be done, I think, is to specify better somehow that some problems have the same reason.

Actually this isn't an intrinsic property of multimodule verification. Similar issues exist for all components, e.g. you tried to investigate reasons of timeouts, some bugs in Weaver can steam from issues in EMG, etc. "Simple" regular expressions can't always help. So, some additional things are required.

I suppose you will think more about this and suggest some nice improvement in unknown reports representation that will allow to identify failure reasons in a more advanced way in comparison with regular expressions. Don't forget about a timeouts classification and either relate the given issue with the corresponding one if it exists, or open a new one, or make this one more generic.

#4 Updated by Ilja Zakharov over 1 year ago

  • Priority changed from High to Urgent

Time to solve it is closing.

#5 Updated by Evgeny Novikov over 1 year ago

  • Tracker changed from Bug to Feature

This doesn't look like a bug.

#6 Updated by Evgeny Novikov over 1 year ago

  • Target version set to 1.0

#7 Updated by Ilja Zakharov over 1 year ago

  • Target version changed from 1.0 to 2.0

#8 Updated by Evgeny Novikov 8 months ago

  • Assignee changed from Alexey Polushkin to Ilja Zakharov

#9 Updated by Ilja Zakharov 6 months ago

  • Status changed from New to Resolved

All subtasks are resolved.

#10 Updated by Evgeny Novikov 5 months ago

  • Status changed from Resolved to Closed

Branch klever-2.0 passed all tests and I merged it to master in 72be796e3 marked as v2.0rc1.

Also available in: Atom PDF