Project

General

Profile

Actions

Feature #6739

closed

Keep and visualize coverage

Added by Evgeny Novikov about 8 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Urgent
Category:
Bridge
Target version:
-
Start date:
02/03/2016
Due date:
% Done:

0%

Estimated time:
Published in build:

Description

Likely we will need to develop our own visualization technique since LCOV functionality will not match all our demands sooner or later.


Files


Related issues 2 (0 open2 closed)

Related to Klever - Feature #8352: Calculate and visualize total code coverageClosedEvgeny Novikov08/15/201710/09/2017

Actions
Blocked by Klever - Feature #6738: Upload coverage if soClosedAlexey Polushkin02/03/2016

Actions
Actions #1

Updated by Evgeny Novikov about 8 years ago

  • Subject changed from Keep and visuilize coverage to Keep and visualize coverage
Actions #2

Updated by Evgeny Novikov over 7 years ago

  • Priority changed from Normal to Urgent

Code coverage is a quite interesting view on verification quality. So when Core will report it Bridge should visualize it nicely.

Actions #3

Updated by Evgeny Novikov over 7 years ago

  • Priority changed from Urgent to High

This issue can't be urgent since its predecessors won't be implemented soon.

Actions #4

Updated by Evgeny Novikov almost 7 years ago

  • Priority changed from High to Urgent

This feature is strictly necessary for the common workflow but we need to wait until the blocking issue will be resolved.

Actions #5

Updated by Alexey Polushkin over 6 years ago

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Actions #6

Updated by Vladimir Gratinskiy over 6 years ago

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

Actions #7

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

The latter is possible always. Regarding the former can't you open new items to the left when there isn't enough space like with very many sub-menus? Also new subdirectories can be shown not after the previous ones but below them like:

subdir1
 subdir2
  subdir3
   ...
    file

Actions #8

Updated by Evgeny Novikov over 6 years ago

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

The latter is possible always. Regarding the former can't you open new items to the left when there isn't enough space like with very many sub-menus? Also new subdirectories can be shown not after the previous ones but below them like:
[...]

One more solution is to reduce menu sizes. Now they look like:

[subdir1     ][subdir2      ][file        ]

You can reduce the width in the following way:
[subdir1][subdir2][file]

But this doesn't solve the issue in general.

Actions #9

Updated by Vladimir Gratinskiy over 6 years ago

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

The latter is possible always. Regarding the former can't you open new items to the left when there isn't enough space like with very many sub-menus? Also new subdirectories can be shown not after the previous ones but below them like:
[...]

If there are more than one subdirectories/files in any dir then it will look like (when you hover subdir 2):

dir
 subdir1
 subdir2
 subdir3
   file_of_subdir2

And now how you want to hover file_of_subdir2 without hovering subdir3 (which is closing subdir2 and opens content of subdir3)? Semantic ui doesn't provide too much options for such menu, so I can't show content of subdir2 over the subdir3.

Actions #10

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

The latter is possible always. Regarding the former can't you open new items to the left when there isn't enough space like with very many sub-menus? Also new subdirectories can be shown not after the previous ones but below them like:
[...]

If there are more than one subdirectories/files in any dir then it will look like (when you hover subdir 2):

[...]

And now how you want to hover file_of_subdir2 without hovering subdir3 (which is closing subdir2 and opens content of subdir3)? Semantic ui doesn't provide too much options for such menu, so I can't show content of subdir2 over the subdir3.

Okay. If the current framework doesn't support this, what about the first and the third solutions (their combination is even better)?

Actions #11

Updated by Vladimir Gratinskiy over 6 years ago

Evgeny Novikov wrote:

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Alexey Polushkin wrote:

Can not open file for coverage if path is too long. I have attached screenshot with this problem

Everything I can do is to add a little bit more space. But you can choose file from the statistic table or make path shorter.

The latter is possible always. Regarding the former can't you open new items to the left when there isn't enough space like with very many sub-menus? Also new subdirectories can be shown not after the previous ones but below them like:
[...]

One more solution is to reduce menu sizes. Now they look like:
[...]
You can reduce the width in the following way:
[...]
But this doesn't solve the issue in general.

I didn't find an option in semantic ui docs to reduce size separately for each item in menu.

Actions #12

Updated by Vladimir Gratinskiy over 6 years ago

The first solution is not supported in semantic ui also.

Actions #13

Updated by Evgeny Novikov over 6 years ago

I have more coverage visualization improvement suggestions:
  1. When the coverage page is open I see an almost empty screen like at the attached screenshot. I suggest to show coverage statistics by default as well as open coverage of a first C source file from directory source files.
  2. Merge first and second columns of a covered source code window. At the moment there are just stub links that don't nothing (there won't be data and its statistics in real life soon). I suggest to have links just some data is available and show it on clicking.
  3. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.
  4. Button Show attributes doesn't work at the coverage page.
  5. Color maps for a both line and function coverage don't look nice because of they are too wide and are placed to far from the covered source code window. It will be better to place them horizontally rather than vertically.
  6. It would be nice to see the number of passes through a source line not only on covering a line number but also on covering lines themselves.
Actions #14

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

The first solution is not supported in semantic ui also.

So nothing works, so, your solution "But you can choose file from the statistic table" looks to be the only choice.

Actions #15

Updated by Vladimir Gratinskiy over 6 years ago

Evgeny Novikov wrote:

I have more coverage visualization improvement suggestions:
  1. When the coverage page is open I see an almost empty screen like at the attached screenshot. I suggest to show coverage statistics by default as well as open coverage of a first C source file from directory source files.

Showing the first source file is not obvious.

  1. Merge first and second columns of a covered source code window. At the moment there are just stub links that don't nothing (there won't be data and its statistics in real life soon). I suggest to have links just some data is available and show it on clicking.

If you don't want the second window, just set the option in settings to hide data.

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

This setting just replaces links to coverage page. So when you set the option, just go to verdict page and then go to coverage page or click the icon "Show coverage without data" on coverage page.

  1. Button Show attributes doesn't work at the coverage page.

Maybe it is because the verification report doesn't have any attributes?

  1. Color maps for a both line and function coverage don't look nice because of they are too wide and are placed to far from the covered source code window. It will be better to place them horizontally rather than vertically.

Have done it horizontally.

  1. It would be nice to see the number of passes through a source line not only on covering a line number but also on covering lines themselves.

Done.

Actions #16

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I have more coverage visualization improvement suggestions:
  1. When the coverage page is open I see an almost empty screen like at the attached screenshot. I suggest to show coverage statistics by default as well as open coverage of a first C source file from directory source files.

Showing the first source file is not obvious.

But it is much more better than showing an almost empty screen. I think that it can be trivially done.

  1. Merge first and second columns of a covered source code window. At the moment there are just stub links that don't nothing (there won't be data and its statistics in real life soon). I suggest to have links just some data is available and show it on clicking.

If you don't want the second window, just set the option in settings to hide data.

Please, read carefully what I wrote. There are three auxiliary columns in covered source code window:
line number (link) - data existence - function declaration coverage.
I suggested to merge the first and the second to reduce space and avoid a lot of stub links that don't work like, say, for error traces.

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

This setting just replaces links to coverage page. So when you set the option, just go to verdict page and then go to coverage page or click the icon "Show coverage without data" on coverage page.

Now I catch this although this is abnormal. Why not have the same page addresses but various implementations? Also there are still a lot of stub links on line numbers (without data they are absolutely useless).

  1. Button Show attributes doesn't work at the coverage page.

Maybe it is because the verification report doesn't have any attributes?

But at least its parent reports have. So they should be shown.

  1. Color maps for a both line and function coverage don't look nice because of they are too wide and are placed to far from the covered source code window. It will be better to place them horizontally rather than vertically.

Have done it horizontally.

  1. It would be nice to see the number of passes through a source line not only on covering a line number but also on covering lines themselves.

Done.

Actions #17

Updated by Alexey Khoroshilov over 6 years ago

Vladimir Gratinskiy wrote:

I didn't find an option in semantic ui docs to reduce size separately for each item in menu.

It seems that menu is not appropriate choice here?

Actions #18

Updated by Alexey Polushkin over 6 years ago

I have got 'Unknown error', when open Coverage statistics. I have attached coverage archive.

Bridge error trace:
Traceback (most recent call last):
File "/home/alexey/klever/klever/bridge/reports/views.py", line 874, in get_coverage_statistic
table = CoverageStatistics(request.POST['report_id']).table_html
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 472, in __init__
self._table_data = self.__get_table_data()
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 574, in __get_table_data
parents[fname]['lines']['color'] = TABLE_STAT_COLOR[color_id]
IndexError: list index out of range

Actions #19

Updated by Vladimir Gratinskiy over 6 years ago

Alexey Khoroshilov wrote:

Vladimir Gratinskiy wrote:

I didn't find an option in semantic ui docs to reduce size separately for each item in menu.

It seems that menu is not appropriate choice here?

OK then, I can just add list of files (not tree) as it were suggested in specification.

Actions #20

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

Alexey Khoroshilov wrote:

Vladimir Gratinskiy wrote:

I didn't find an option in semantic ui docs to reduce size separately for each item in menu.

It seems that menu is not appropriate choice here?

OK then, I can just add list of files (not tree) as it were suggested in specification.

My suggestion is to remove this button at all since required files can be found using a source tree representation for coverage statistics, especially when it will have some more advanced filters and show/hide actions.

Actions #21

Updated by Vladimir Gratinskiy over 6 years ago

Files tree button was removed. First found file and coverage statistic are shown by default. Columns "line number" and "has data" were merged. All attributes are shown.

Concerning different links to coverage with/without data - there is link on coverage page to minimize/maximize data.

Actions #22

Updated by Evgeny Novikov over 6 years ago

Did you fix this:
Alexey Polushkin wrote:

I have got 'Unknown error', when open Coverage statistics. I have attached coverage archive.

Bridge error trace:
Traceback (most recent call last):
File "/home/alexey/klever/klever/bridge/reports/views.py", line 874, in get_coverage_statistic
table = CoverageStatistics(request.POST['report_id']).table_html
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 472, in __init__
self._table_data = self.__get_table_data()
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 574, in __get_table_data
parents[fname]['lines']['color'] = TABLE_STAT_COLOR[color_id]
IndexError: list index out of range

After that the issue can be marked as resolved and all related branches can be merged to master.

Actions #23

Updated by Vladimir Gratinskiy over 6 years ago

Evgeny Novikov wrote:

Did you fix this:
Alexey Polushkin wrote:

I have got 'Unknown error', when open Coverage statistics. I have attached coverage archive.

Bridge error trace:
Traceback (most recent call last):
File "/home/alexey/klever/klever/bridge/reports/views.py", line 874, in get_coverage_statistic
table = CoverageStatistics(request.POST['report_id']).table_html
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 472, in __init__
self._table_data = self.__get_table_data()
File "/home/alexey/klever/klever/bridge/reports/coverage.py", line 574, in __get_table_data
parents[fname]['lines']['color'] = TABLE_STAT_COLOR[color_id]
IndexError: list index out of range

After that the issue can be marked as resolved and all related branches can be merged to master.

Fixed.

Actions #24

Updated by Evgeny Novikov over 6 years ago

Evgeny Novikov wrote:

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

The latter wasn't implemented. Why we need a list of choices containing of two Boolean variants?

Actions #25

Updated by Evgeny Novikov over 6 years ago

Evgeny Novikov wrote:

Evgeny Novikov wrote:

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

The latter wasn't implemented. Why we need a list of choices containing of two Boolean variants?

In addition I suppose to switch the default value from "with data" to "without data" since there likely won't be any data in near future.

Actions #26

Updated by Vladimir Gratinskiy over 6 years ago

Evgeny Novikov wrote:

Evgeny Novikov wrote:

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

The latter wasn't implemented. Why we need a list of choices containing of two Boolean variants?

I have to delete button "hide/show" data on coverage page to implement this feature (hide data on page refresh if user changed settings). Everybody agree?
I have replaced dropdown select with checkbox like for error rtace assumptions and set it to false by default.

Actions #27

Updated by Evgeny Novikov over 6 years ago

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Evgeny Novikov wrote:

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

The latter wasn't implemented. Why we need a list of choices containing of two Boolean variants?

I have to delete button "hide/show" data on coverage page to implement this feature (hide data on page refresh if user changed settings). Everybody agree?

I don't ask to do this anymore. I just said about the second part of this feature: "Also I suggest to make it like for error trace assumptions.".

I have replaced dropdown select with checkbox like for error rtace assumptions and set it to false by default.

It was likely done and many excessive things were removed (before this setting was likely intended for more than 2 choices but it turned out that it always has just 2 ones). But I got the following exception when migrating a database:

manage.py@bridge > migrate
bash -cl "/home/novikov/.pyenv/versions/virtual-env-3.6.1/bin/python /home/novikov/work/pycharm-2017.1.1/helpers/pycharm/django_manage.py migrate /home/novikov/work/klever/bridge" 
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0010_auto_comparison_user, 0011_reportcomponent_coverage_arch in reports).
To fix them run 'python manage.py makemigrations --merge'

Please, do all related improvements and fixes in branch coverage_and_bridge since it combines everything related with coverage.

Actions #28

Updated by Evgeny Novikov over 6 years ago

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Evgeny Novikov wrote:

  1. A user setting to hide a coverage data doesn't work. Also I suggest to make it like for error trace assumptions.

The latter wasn't implemented. Why we need a list of choices containing of two Boolean variants?

I have replaced dropdown select with checkbox like for error rtace assumptions and set it to false by default.

It was likely done and many excessive things were removed (before this setting was likely intended for more than 2 choices but it turned out that it always has just 2 ones). But I got the following exception when migrating a database:
[...]

Likely this conflict exists after merging branch feature_8007 to master and master to branch coverage_and_bridge. Since feature_8007 is already in master migrations from coverage_and_bridge should be rebased.

Actions #29

Updated by Vladimir Gratinskiy over 6 years ago

Rebased.

Actions #30

Updated by Evgeny Novikov over 6 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF