Feature #8568
closed
Added by Evgeny Novikov about 7 years ago.
Updated almost 5 years ago.
Category:
Tasks generation
Description
After Ilja supported reusing SA and EMG results one of the most time consuming and redundant operation is processing the same models for various verification objects and rule specifications. Moreover several file transformations, e.g. preprocessing of CIF output files, should be avoided.
- Assignee changed from Ilja Zakharov to Evgeny Novikov
- Priority changed from High to Urgent
- Target version set to 3.0
Weaver became a real bottleneck. It does too much I/O work most of which is redundant. For instance, its wall time exceeds its CPU time in ~6 times!
- Subject changed from Reuse processed models to Optimize weaving
- Description updated (diff)
- Priority changed from Urgent to High
- Target version deleted (
3.0)
Let's do minor optimizations one day later.
- Blocked by Feature #6731: Pass source code just through C-backend if aspects are empty added
- Priority changed from High to Urgent
- Target version set to 3.0
This issue became urgent after Weaver started to obtain cross references for models with Clade. Although, it is possible to essentially speed up this by launching builds and Clade in parallel, but it is just a workaround that can cause very unpleasant consumption of computational resources. So, I suggest to implement some trivial caching to store immutable results of both weaving and cross reference calculation.
- Status changed from New to Closed
I implemented the feature in branch weaver-cache that I merged to master in 499b949a6.
The optimization speed ups Weaver considerably, so that it becomes even faster than as it was before we added support for so called additional sources and cross references. On powerful machines with fast disks it is not very noticeable, but, say, on OpenStack virtual machines one will get considerable improvement. Besides, it affects those jobs that have rather simple tasks much while for jobs with complicated tasks the absolute difference will be not so big.
One important note! By using option "Original C source files" (it is default for the production mode) for "Code coverage details" when starting job decisions, one can speed up Weaver by an order of magnitude. Of course, there will be neither additional sources except for original ones nor cross references and highlights for them.
Also available in: Atom
PDF