Feature #9412
closedSupport more advanced calculation of total verdicts and similarity
100%
Description
- Imagine there is no associated marks for a given report (Unsafe/Safe/Unknown).
- The expert needs to create a new mark.
- It becomes confirmed automatically for a given report and may be associated automatically with some other reports.
- Then the expert proceeds to a next report.
- If there is no associated marks, the expert should do the same (1-3).
- If there are some automatically associated marks and the expert finds one or more of them appropriate, the expert should confirm corresponding associations.
- Otherwise, the expert must create a new mark like in previous use cases (1-3, 5).
In any case there will be at least one confirmed association (in ideal, there should be the only confirmed association) for each report. And if there are confirmed associations, Bridge should not treat automatically associated marks when calculating total verdict and total similarity. If there is no confirmed associations, Bridge should behave as it does now.
PS: Let's exclude associations with 0% similarity from calculating total verdicts and similarity (inspired by #9413). This will improve a picture for automatically associated marks a bit since there will be less incompatible marks. But in general this will not help while the primary suggested feature will do. The pitfall may be confirmed associations with 0% similarity as it should be both used and not used in calculating total values according with various rules. I can hardly imagine cases when one will need to confirm associations with 0% similarity, but let's respect manual efforts (confirmation) first of all. So, let's calculate total values for confirmed associations with 0% similarity.
PPS: Let's exclude total similarity at all as like some other totals it is ambiguous. Some people likes one its semantics, other people prefers another one. But nobody will suffer much if there will be no total similarity at all since it is primarily useful just when considering particular error traces and their associated marks.
PPPS: I suggest to allow for mark authors to disable further mark changes. This will disable the only case (removing marks or changing them considerably) when somebody can affect evaluation of verification results made by mark authors. Of course, this does not include 3rd party marks which associations are confirmed.
Updated by Ilja Zakharov almost 6 years ago
- Related to Feature #9413: Set total verdict and total similarity more precisely on base of automatic marks association added
Updated by Evgeny Novikov almost 6 years ago
Indeed, I dislike the suggested approach as it contradicts the primary workflow. Nobody prevents you to unconfirm new associatead marks - this way suits well the primary workflow. But I agree that often one is too lazy to do such the job.
I already suggested another way to reach the same result using other means. These means include protecting jobs from associations with all existing marks. So, you will need to create marks for all reports of protected jobs, and nobody will be able to affect your evaluation. One another mean to reach this is ability to freeze marks by their authors. Note that this use case forbids other issues, e.g. when you confirm some associations, but then one will change associated marks, your confirmations will be cancelled.
Updated by Evgeny Novikov almost 6 years ago
We discussed this issue more. Now I agree that the improvement suggested by Ilja is good. So, the primary workflow will look as follows. Imagine there is no associated marks for a given error trace. You create a new mark. It becomes confirmed automatically. Then you go to a next error trace. If there is no associated marks, you should do the same. If there are some automatically associated marks and you find one of them appropriate, you should confirm a corresponding association. Otherwise, you will create a new mark like in previous use cases. In any case there will be at least one confirmed association (in ideal, there should be the only confirmed association). And if there are confirmed associations, Bridge should not treat automatically associated marks when calculating total verdict and total similarity. If there is no confirmed associations, Bridge should behave as it does now.
Updated by Pavel Andrianov almost 6 years ago
I support the issue: incompatible marks are very distractive. One minor comment is that several confirmed associations may be present. This is not a general case, when we want to emphasize several marks as confirmed, but Bridge should not fail in this case.
Updated by Evgeny Novikov almost 6 years ago
Pavel Andrianov wrote:
I support the issue: incompatible marks are very distractive. One minor comment is that several confirmed associations may be present. This is not a general case, when we want to emphasize several marks as confirmed, but Bridge should not fail in this case.
Bridge should not fail in any case. I also specified that there will be at least one confirmed association for each report. But there may be more ones. Perhaps, it is important to note, that in case when there are several confirmed associations Bridge should calculate total verdicts and similarity like when there are several automatic associations without confirmed ones.
Updated by Evgeny Novikov almost 6 years ago
- Subject changed from Do not consider an error trace with manual confirmations as "incompatible marks" to Support more advanced calculation of total verdicts and similarity
- Description updated (diff)
Updated by Evgeny Novikov almost 6 years ago
- Related to deleted (Feature #9413: Set total verdict and total similarity more precisely on base of automatic marks association)
Updated by Evgeny Novikov almost 6 years ago
- Has duplicate Feature #9413: Set total verdict and total similarity more precisely on base of automatic marks association added
Updated by Evgeny Novikov almost 6 years ago
- Related to Feature #8335: Similarity management added
Updated by Evgeny Novikov almost 6 years ago
- Related to Feature #8338: Add ability to specify similarity threshold added
Updated by Vladimir Gratinskiy over 5 years ago
PPPS: I suggest to allow for mark authors to disable further mark changes.
Who is the author of a mark - the user created the first version or the last change author?
Updated by Evgeny Novikov over 5 years ago
Vladimir Gratinskiy wrote:
PPPS: I suggest to allow for mark authors to disable further mark changes.
Who is the author of a mark - the user created the first version or the last change author?
I think the former is much better.
Updated by Vladimir Gratinskiy over 5 years ago
- Due date set to 07/10/2019
- Status changed from New to Resolved
- % Done changed from 0 to 100
Implemented in bridge-3.0.
1) 0% similarity will be excluded from total verdict calculation if mark threshold is greater than 0%. Otherwise it will be excluded if there is already another confirmed mark. I've added new column and its filter for associations table "is associated", that shows if mark association is used in total verdict calculation.
2) "total similzrity" was deleted from everywhere.
3) Now not only managers, but mark authors also can freeze their marks.