Project

General

Profile

Actions

Feature #562

open

Stats server has to provide us with problems comparison

Added by Evgeny Novikov over 13 years ago. Updated almost 13 years ago.

Status:
Open
Priority:
Normal
Category:
Statistics server
Start date:
11/16/2010
Due date:
% Done:

0%

Estimated time:
Published in build:

Description

I have thought about this at the beginning of stats comparison implementation but then it wasn't the task of high priority. From now it'll be very useful because of this can show the progress of our tool development more precisely. For example how time limits becomes memory limits and vise versa and so on.
The technical aspect is to extend comparison with conversions like P1->P2 where P1 is some first kind problem from a set of problems of all tools, P2 is some second kind problem of a given set. These conversions belongs to the general Unknown->Unknown conversion and show/hide must be available for them (by default conversions with different problems are shown, with the same problem are hide).
The issue comes from middle regression testing. After adding of problems patterns for build command extractor and some other changes the comparison with diff shows 66 real changes while the stats server shows just 3 changes. This is because 3 changes belong to the Unknown->Safe conversion while 63 changes belongs to the general Unknown->Unknown conversion that has subclasses:
  • Time_limit->Out_of_memory
  • ''->Compile error ('' means unmatched problem)
  • ''->No Makefile or Kbuild file
  • Exception,Parsing errors->Exception
  • Exception,Parsing errors->Out_of_memory

Related issues 1 (1 open0 closed)

Related to Linux Driver Verification - Bug #548: Information about problematic launches can't make its way to results databaseNew11/15/2010

Actions
Actions

Also available in: Atom PDF