Feature #7902
closedSupport medium weight verification results
100%
Description
Missing all details for safes isn't actually very helpful since users often would like to analyze some meta information about them at least while it gives much less as say missing all intermediate reports and logs from the optimizations point of view. For instance, one can need to see corresponding attributes and statistics (e.g. distribution by rule specifications), to see consumed CPU time (#7897), etc. Also one would like to get input files for verifiers for safes like for unsafes if the corresponding option is activated.
That is why I suppose to support medium weight verification results for which the same information as for unsafes will be stored for safes and to use this mode by default for production.
Most likely later we will get rid of lightweight verification results support at all.
Files
Updated by Evgeny Novikov over 7 years ago
Also it would be helpful to have a button to convert not only to lightweight verification results but for medium weight ones. You can experiment with the job archive from here. It is interesting to compare sizes of archives with medium weight and lightweight verification results as well as their download and upload times.
Updated by Vladimir Gratinskiy over 7 years ago
- Due date set to 01/26/2017
- Status changed from New to Resolved
- % Done changed from 0 to 100
Implemented in branch "feature_7902". The button "Collapse reports" can be clicked twice - when the job is full-weight and when it medium-weight. First click will convert the job to medium-weight, the second - to lightweght. I've slao renamed option "lightweightness" in core configuration to "weight" which is one of 3 options: '0' for full-weight, '1' for medium-weight and '2' for lightweght jobs. Also I have optimised uploading reports.
Uploading the job can take a lot of time, so now I don't allow to upload jobs while there are pending or processing jobs in the system.
And I've updated all migrations. Old results can't be saved as jobs will have new format.
Updated by Evgeny Novikov over 7 years ago
Downloading and uploading archives of jobs with lightweight verification results using production servers already works very fast. For jobs with usual verification results this isn't the case, so this optimization is welcome.
Please merge branch fix_7897 to branch feature_7902 so that I will be able to test them both. I expect there will be some nontrivial conflicts, e.g. in migrations for users.
Also it would be great if you will measure and write here characteristics mentioned in https://forge.ispras.ru/issues/7902#note-1.
Updated by Vladimir Gratinskiy over 7 years ago
- File download-stat.png download-stat.png added
Evgeny Novikov wrote:
Also it would be great if you will measure and write here characteristics mentioned in https://forge.ispras.ru/issues/7902#note-1.
I have attached the screenshot with comparison. Initialising generators and streaming job archives content is not included in download time as it is processed by Django server funcitons and I can't calculate time for it.
Updated by Vladimir Gratinskiy over 7 years ago
I have merged the branch fix_7897 and update migrations so that if your DB synchronized with feature_7902 before the merge you can migrate after it. But it would be better to recreate migrations.
Updated by Vladimir Gratinskiy over 7 years ago
Uploading medium archives can take too much time especially if there are a lot of unsafe/safe/unknown marks. So if medium jobs will be default format then uploading jobs while there are processing/pending jobs is not good idea.
Updated by Vladimir Gratinskiy over 7 years ago
I will merge branch fix_7891 to feature_7902 too and then finish #7891.
Updated by Evgeny Novikov over 7 years ago
- Status changed from Resolved to Closed
After all works perfectly. I merged the branch to master in 61cc1ca.