Klever

Is this threshold associated with a mark? Imagine a situation: I am analyzing one launch and am creating marks with threshold 100%. Other person is analyzing other launch, but on the same machine and creating marks with threshold 50%. His marks are successfully applied to my job (launch) results, but I do not want such low threshold. Is there any protection?

Pavel Andrianov wrote:

Is this threshold associated with a mark? Imagine a situation: I am analyzing one launch and am creating marks with threshold 100%. Other person is analyzing other launch, but on the same machine and creating marks with threshold 50%. His marks are successfully applied to my job (launch) results, but I do not want such low threshold. Is there any protection?

Good catch! This is a well known issue I thought about even when I wrote the specification. But it can be considered as a feature which can give quite much when playing carefully (e.g. one can find out duplicates of warnings or marks, suspicious reports or expert efforts and so on).

Indeed it isn't related with the similarity and its threshold - it is just related with the fact that there isn't any protection from associating third-party marks with your verification results even if they are private, i.e. other users can't see them and, say, create marks for them.

But as I already specified when discussing #8335 laziness can result in various difficulties anyway. From the other side I noticed that it can be not so bad if you will quickly "forget" about analyzed verification results as it usually happens.

Verification results protection should be implemented independently from this issue as well as #8335 since both of them doesn't help in this way in the general case (imagine that one can break your perfect statistics by creating malicious marks with similarity 100%). If you still think that it is useful for you, please, open another issue.

Do we have a hope for the feature to be implemented in the nearest version?

Do you need this in case when #9412 will be implemented?

Yes, we still need it even with #9412

Pavel, what do you think? Will you agree that #9412 is a more proper way to evaluate verification results in general? This feature looks more like a cheat that can be supported and that will help in some cases to avoid manual work, but in many cases it will be wrong (both errors of first and second kind).

I do not think it is a cheat. I would not like to resolve conflicts between 17% similar mark and 22% one, if, for example, they are incompatible. Also I do not see any wrong cases if we do not apply marks to a not similar traces. The only difference is less automatic applied marks.

Pavel Andrianov wrote:

I do not think it is a cheat. I would not like to resolve conflicts between 17% similar mark and 22% one, if, for example, they are incompatible. Also I do not see any wrong cases if we do not apply marks to a not similar traces. The only difference is less automatic applied marks.

With #9412 you will need to either confirm one of existing associations or create a new mark for each report. And then you will have constant statistics unless somebody will delete marks having confirmed associations with reports of your job. I suggest to have an option to forbid this by mark authors.

Regarding your case, do you think that 17% is worse than 22%? Indeed, both associations may be irrelevant. Without investigation and manual evaluation no algorithm can provide you valid results.

Regarding your case, do you think that 17% is worse than 22%?

Yes, both of them are irrelevant, so I do not like to open them, find differences or investigate them in any other way. I know, that similarity < 50% is useless. So, why we can not use this knowledge to prompt Klever. Let hide such marks and not distract the user.

Without investigation and manual evaluation no algorithm can provide you valid results.

In general case, yes. Anyway, there are other use cases, when a user needs to have only general overview. For example, several jobs with many unsafes and marks. Here manual confirmation will take a lot of time. With this feature a user may choose high similarity threshold to avoid automatic incompatible marks. The final results will be not so precise as with manual confirmation, but they will be much quicker.