Project

General

Profile

Feature #8568

Optimize weaving

Added by Evgeny Novikov over 2 years ago. Updated 4 months ago.

Status:
Closed
Priority:
Urgent
Category:
Tasks generation
Target version:
Start date:
11/13/2017
Due date:
% Done:

0%

Estimated time:
Published in build:

Description

After Ilja supported reusing SA and EMG results one of the most time consuming and redundant operation is processing the same models for various verification objects and rule specifications. Moreover several file transformations, e.g. preprocessing of CIF output files, should be avoided.


Related issues

Blocked by Klever - Feature #6731: Pass source code just through C-backend if aspects are emptyClosed02/03/2016

Actions

History

#1

Updated by Evgeny Novikov almost 2 years ago

  • Target version set to 3.0
  • Priority changed from High to Urgent
  • Assignee changed from Ilja Zakharov to Evgeny Novikov

Weaver became a real bottleneck. It does too much I/O work most of which is redundant. For instance, its wall time exceeds its CPU time in ~6 times!

#2

Updated by Evgeny Novikov over 1 year ago

  • Description updated (diff)
  • Subject changed from Reuse processed models to Optimize weaving
#3

Updated by Evgeny Novikov over 1 year ago

  • Target version deleted (3.0)
  • Priority changed from Urgent to High

Let's do minor optimizations one day later.

#4

Updated by Evgeny Novikov about 1 year ago

  • Blocked by Feature #6731: Pass source code just through C-backend if aspects are empty added
#5

Updated by Evgeny Novikov 4 months ago

  • Target version set to 3.0
  • Priority changed from High to Urgent

This issue became urgent after Weaver started to obtain cross references for models with Clade. Although, it is possible to essentially speed up this by launching builds and Clade in parallel, but it is just a workaround that can cause very unpleasant consumption of computational resources. So, I suggest to implement some trivial caching to store immutable results of both weaving and cross reference calculation.

#6

Updated by Evgeny Novikov 4 months ago

  • Status changed from New to Closed

I implemented the feature in branch weaver-cache that I merged to master in 499b949a6.

The optimization speed ups Weaver considerably, so that it becomes even faster than as it was before we added support for so called additional sources and cross references. On powerful machines with fast disks it is not very noticeable, but, say, on OpenStack virtual machines one will get considerable improvement. Besides, it affects those jobs that have rather simple tasks much while for jobs with complicated tasks the absolute difference will be not so big.

#7

Updated by Evgeny Novikov 4 months ago

One important note! By using option "Original C source files" (it is default for the production mode) for "Code coverage details" when starting job decisions, one can speed up Weaver by an order of magnitude. Of course, there will be neither additional sources except for original ones nor cross references and highlights for them.

#8

Updated by Evgeny Novikov 4 months ago

This works since 1ec20b120 to master.

Also available in: Atom PDF