Klever

You should change global job role (or set "Observer" role for me). Now I don't have an access to these reports.

I've updated global job role

Maybe you want to use another error trace convertion and comparison functions ("call_stack" for example). "call_forests_compare" contains call trees with leaves that have "note". Your error traces don't have at least one note.

"callstack_compare" can't associate mark on http://ldvstore:8998/reports/unsafe/73899/ error trace even if mark was created exactly on this trace.
Is it require to add "note" at last error edge into witness also for callstack comparator?

Anton Vasilyev wrote:

"callstack_compare" can't associate mark on http://ldvstore:8998/reports/unsafe/73899/ error trace even if mark was created exactly on this trace.

You should create new mark with "call_stack" convert and "callstack_compare" comparison functions.

Is it require to add "note" at last error edge into witness also for callstack comparator?

No, callstack comparator needs only warning.

Evgeny Novikov wrote:

Please, treat edges with warnings exactly as edges with notes. Before we didn't have such issues since there always were edges with notes under edges with warnings.

It will not fix issue for this error traces as warning is not in enter function edge. So, again, either change functions, or fix error traces.

Fixed in branch "forests_with_callbacks".

Evgeny Novikov wrote:

This helped. Should we propose to extend manager tools or clear cache automatically on each functions update?

Functions bodies are not checked now so clearing cache automatically can't be done. Manager tools are for clearing unused data left after some bridge work. So clearing this cache should be done by migrations.

I've added the new migration that deletes needed cache.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

This helped. Should we propose to extend manager tools or clear cache automatically on each functions update?

Functions bodies are not checked now so clearing cache automatically can't be done. Manager tools are for clearing unused data left after some bridge work. So clearing this cache should be done by migrations.

But during population you track somehow what functions changed and report this. Why you can't clear corresponding caches automatically? If this is hard, you can clear all corresponding caches after each population which is performed quite rare. Migrations will be a bit hard to maintain because of models don't changed and one should keep in mind that after each function change it is necessary to create corresponding migration which is also nontrivial. Perhaps functions should be a part of models but this is another story.

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

This helped. Should we propose to extend manager tools or clear cache automatically on each functions update?

Functions bodies are not checked now so clearing cache automatically can't be done. Manager tools are for clearing unused data left after some bridge work. So clearing this cache should be done by migrations.

But during population you track somehow what functions changed and report this. Why you can't clear corresponding caches automatically? If this is hard, you can clear all corresponding caches after each population which is performed quite rare. Migrations will be a bit hard to maintain because of models don't changed and one should keep in mind that after each function change it is necessary to create corresponding migration which is also nontrivial. Perhaps functions should be a part of models but this is another story.

During population I track just names as only names are saved in DB. How you think I can check if body changed? I need to collect also bodies of all functions called inside convert function, and bodies of functions called inside these functions and so on. Also I need to collect content of classes. But I need to restrict depth of checking by bridge functions and don't check any library functions.
And I didn't change convert function, I changed method of classes which called somewhere deep in callstack.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

This helped. Should we propose to extend manager tools or clear cache automatically on each functions update?

Functions bodies are not checked now so clearing cache automatically can't be done. Manager tools are for clearing unused data left after some bridge work. So clearing this cache should be done by migrations.

But during population you track somehow what functions changed and report this. Why you can't clear corresponding caches automatically? If this is hard, you can clear all corresponding caches after each population which is performed quite rare. Migrations will be a bit hard to maintain because of models don't changed and one should keep in mind that after each function change it is necessary to create corresponding migration which is also nontrivial. Perhaps functions should be a part of models but this is another story.

During population I track just names as only names are saved in DB. How you think I can check if body changed? I need to collect also bodies of all functions called inside convert function, and bodies of functions called inside these functions and so on. Also I need to collect content of classes. But I need to restrict depth of checking by bridge functions and don't check any library functions.
And I didn't change convert function, I changed method of classes which called somewhere deep in callstack.

Now I understand why converting and comparing changes after source code update even without population. But I can't understand why you can't clear caches when populating. Let's clear these caches for each population independently on whether some new functions were created or some existing functions were removed.

Evgeny Novikov wrote:

I merged the branch to master again in 1ed126a. One needs to populate databases to take changes.

BTW, most likely there is a tricky issue. If you change somehow existing comparison criterion this doesn't affect already established bindings between unsafes and marks. This isn't a bug at all, since these bindings were established, analyzed and confirmed by users before. One can't just recalculate them when populating/migrating databases, since after that you can miss some of them or add new ones. But further users can be surprised why some bindings exist although updated/fixed comparison criterion shouldn't behave in such the way. First users should understand this and take care or this themselves. Second, if I remember properly, we designed functions so that if there is some change, then it should take new names, e.g. function_2, function_3, etc. It can be a bit inconvenient for developers and likely we will always forget about this rule but users will explicitly know why some bindings exist and some doesn't.

Population deletes all functions that doesn't exist. It would delete all marks that have links to these functions. And deleting marks would delete all connections with reports.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I merged the branch to master again in 1ed126a. One needs to populate databases to take changes.

BTW, most likely there is a tricky issue. If you change somehow existing comparison criterion this doesn't affect already established bindings between unsafes and marks. This isn't a bug at all, since these bindings were established, analyzed and confirmed by users before. One can't just recalculate them when populating/migrating databases, since after that you can miss some of them or add new ones. But further users can be surprised why some bindings exist although updated/fixed comparison criterion shouldn't behave in such the way. First users should understand this and take care or this themselves. Second, if I remember properly, we designed functions so that if there is some change, then it should take new names, e.g. function_2, function_3, etc. It can be a bit inconvenient for developers and likely we will always forget about this rule but users will explicitly know why some bindings exist and some doesn't.

Population deletes all functions that doesn't exist. It would delete all marks that have links to these functions. And deleting marks would delete all connections with reports.

Good to know too. I hope that everybody understood me properly. I didn't suggest to remove functions - just to add new "versions" of these functions. So all existing marks and bindings will remain but one will see that they were created using old "versions" of functions.

The next question. If there is knowledge with what functions converted (pattern) error traces were created? If yes and they can be deleted when corresponding conversion functions will be deleted, then when one will edit them manually this binding should be broken to avoid deletion of semi-manually created marks.

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I merged the branch to master again in 1ed126a. One needs to populate databases to take changes.

BTW, most likely there is a tricky issue. If you change somehow existing comparison criterion this doesn't affect already established bindings between unsafes and marks. This isn't a bug at all, since these bindings were established, analyzed and confirmed by users before. One can't just recalculate them when populating/migrating databases, since after that you can miss some of them or add new ones. But further users can be surprised why some bindings exist although updated/fixed comparison criterion shouldn't behave in such the way. First users should understand this and take care or this themselves. Second, if I remember properly, we designed functions so that if there is some change, then it should take new names, e.g. function_2, function_3, etc. It can be a bit inconvenient for developers and likely we will always forget about this rule but users will explicitly know why some bindings exist and some doesn't.

Population deletes all functions that doesn't exist. It would delete all marks that have links to these functions. And deleting marks would delete all connections with reports.

Good to know too. I hope that everybody understood me properly. I didn't suggest to remove functions - just to add new "versions" of these functions. So all existing marks and bindings will remain but one will see that they were created using old "versions" of functions.

It is bad as functions are complicated and take a lot of code.