https://forge.ispras.ru/https://forge.ispras.ru/favicon.ico?16490126692017-04-24T12:45:23ZOpen-Source ProjectsKlever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=302382017-04-24T12:45:23ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>BTW, the same problem is for several sub-jobs.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=306492017-06-22T18:34:12ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Assignee</strong> deleted (<del><i>Ilja Zakharov</i></del>)</li><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>High</i></li></ul><p>There are too many very high priority issues. This one can be done after really important features will be supported.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=315882017-09-20T08:45:34ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Assignee</strong> set to <i>Ilja Zakharov</i></li><li><strong>Priority</strong> changed from <i>High</i> to <i>Urgent</i></li><li><strong>Target version</strong> set to <i>1.0</i></li></ul><p>In addition to the requested improvements it will be necessary to fix progress calculation at all (that was mentioned in <a class="issue tracker-4 status-6 priority-6 priority-high2 closed" title="Feature: Restore progress calculation (Rejected)" href="https://forge.ispras.ru/issues/8444">#8444</a>).</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=315892017-09-20T08:57:25ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>Ilja suggests to change the progress API between Bridge and Core so that there will be additional report fields intended just for progress calculation and there shouldn't any references to concrete Core component names (something similar to <a class="issue tracker-1 status-6 priority-6 priority-high2 closed" title="Bug: Names of components have been changed and tests and hardcoded names in Bridge should be updated (Rejected)" href="https://forge.ispras.ru/issues/8442">#8442</a>).</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318232017-10-10T12:40:00ZIlja Zakharovilja.zakharov@ispras.ru
<ul></ul><p>To restore functionality of progress evaluation I propose to do the following changes in Bridge first:<br />1) Implement a separate request to Bridge (not as a report) with the following data: {<br />"total tasks to be generated": 11111,<br />"tasks failed": 122,<br />"tasls solved": 500,<br />"average wall time to finish": 3766,<br />"wall time spend on solution": 2300 <br />}<br />This request Core can do several times during the solution. <br />If Core has several sub-jobs it will not send any data at all or do it as there is the only sub-job.</p>
<p>2) Bridge should just store and visualize the data as is calculating the percentage as (failed + solved) * 100/total.</p>
<p>3) If Core provided data to Bridge and the job has PROCESSING status Bridge should send received data to scheduler during service/get_jobs_and_tasks request adding the following section: {<br />....<br />"jobs progress": {<br />"job_id": {data as is sent by Core}<br />}<br />}<br />Addiction of the section can be done only on update the data to economy traffic.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318472017-10-11T09:05:25ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>Ilja Zakharov wrote:</p>
<blockquote>
<p>To restore functionality of progress evaluation I propose to do the following changes in Bridge first:<br />1) Implement a separate request to Bridge (not as a report) with the following data: {<br />"total tasks to be generated": 11111,<br />"tasks failed": 122,<br />"tasls solved": 500,<br />"average wall time to finish": 3766,<br />"wall time spend on solution": 2300 <br />}</p>
</blockquote>
<p>Minor improvements:<br /> "tasks failed" -> "failed tasks" <br /> "tasls solved" -> "solved tasks" <br /> "average wall time to finish" -> "expected time for solving tasks" <br /> "wall time spend on solution" -> "elapsed time for solving tasks"</p>
<blockquote>
<p>This request Core can do several times during the solution.</p>
</blockquote>
<p>My suggestion is to send just changed values since, say, "total tasks to be generated" will not ever change.</p>
<blockquote>
<p>If Core has several sub-jobs it will not send any data at all or do it as there is the only sub-job.</p>
</blockquote>
<p>So, Bridge should expect that there can be no progress reports for some verification jobs.</p>
<blockquote>
<p>2) Bridge should just store and visualize the data as is calculating the percentage as (failed + solved) * 100/total.</p>
</blockquote>
<p>I suggest the following formula: 100 * solved / (total - failed). This should be calculated just if failed < total. Otherwise if failed = total progress can be hidden because of tasks generation/solution finishes.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318582017-10-11T14:31:49ZIlja Zakharovilja.zakharov@ispras.ru
<ul></ul><blockquote>
<p>My suggestion is to send just changed values since, say, "total tasks to be generated" will not ever change.</p>
</blockquote>
<p>I disagree with this point. Since Bridge does not need doing any complicated calculation, let's allow changing the numbers. However, I am not sure that we will change them, but anyway such artificial restrictions are really not necessary.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318592017-10-12T06:37:41ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>Ilja Zakharov wrote:</p>
<blockquote><blockquote>
<p>My suggestion is to send just changed values since, say, "total tasks to be generated" will not ever change.</p>
</blockquote>
<p>I disagree with this point. Since Bridge does not need doing any complicated calculation, let's allow changing the numbers. However, I am not sure that we will change them, but anyway such artificial restrictions are really not necessary.</p>
</blockquote>
<p>I didn't restrict any changes, although it isn't clear. I just suggested to send data incrementally, i.e. send just changed values and likely even after some configurable period of time. For instance, users can request to update a progress just each 30 seconds or each 5 minutes.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318802017-10-16T07:52:15ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>Ilja Zakharov wrote:</p>
<blockquote>
<p>If Core has several sub-jobs it will not send any data at all or do it as there is the only sub-job.</p>
</blockquote>
<p>I suggest a quite simple for implementation and useful for users approach to evaluate a progress for jobs with sub-jobs. Like with task let's evaluate the number of sub-jobs (total, solved, failed) and an average wall time spent on their solution. So, Bridge will show a progress of sub-jobs solution rather than tasks solution. Regarding to data there can be following additional fields:<br /><pre>
{
"sub-jobs to be solved": 50,
"failed sub-jobs": 5,
"solved sub-jobs": 25,
"expected time for solving sub-jobs": 3766,
"elapsed time for solving sub-jobs": 2300
}
</pre></p>
<p>Also, I suggest to name "total tasks to be generated" as "tasks to be generated".</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=318822017-10-16T08:37:02ZIlja Zakharovilja.zakharov@ispras.ru
<ul></ul><blockquote>
<p>Regarding to data there can be following additional fields</p>
</blockquote>
<p>Ok, lets do it also. I would propose to send the data within the same request but make it possible to either attach either data about the tasks and sub-jobs, only about sub-jobs or only about tasks. Because corresponding information core will likely calculate indifferent places.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=319662017-10-30T07:30:30ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>We need a specification that will conclude this discussion and describe all new requests and their semantics in all details.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=319862017-10-31T11:15:05ZIlja Zakharovilja.zakharov@ispras.ru
<ul></ul><p>Lets discuss it here: <a class="external" href="https://goo.gl/H2WFsw">https://goo.gl/H2WFsw</a>.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=319962017-11-01T13:46:43ZIlja Zakharovilja.zakharov@ispras.ru
<ul></ul><p>Added issue <a class="issue tracker-4 status-5 priority-6 priority-high2 closed behind-schedule" title="Feature: Untie coverage report from any components name (Closed)" href="https://forge.ispras.ru/issues/8536">#8536</a> as blocking since I am doing progress calculation on base of refactoring of Job.py which I cannot finish without separate total coverage request.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=320232017-11-07T16:22:34ZIlja Zakharovilja.zakharov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul><p>The updated progress implementation is available in branch 8149-new-progress. I will perform additional tests, but for simple examples all work nicely for both jobs with subjobs and without them.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=320242017-11-08T11:35:02ZIlja Zakharovilja.zakharov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Open</i></li></ul><p>Decided to update the implementation.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=320552017-11-10T08:21:20ZIlja Zakharovilja.zakharov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Resolved</i></li></ul><p>The final implementation is in branch "8149-new-progress". I see no any explicit bugs there at the moment.</p> Klever - Feature #8149: Think on proper progress evaluation when several jobs are solved at once or/and much computational resources are availablehttps://forge.ispras.ru/issues/8149?journal_id=321002017-11-14T10:38:24ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li></ul><p>I merged the branch to master in <a class="changeset" title="Merge branch '8149-new-progress'" href="https://forge.ispras.ru/projects/klever/repository/331/revisions/459f75e70c9d13331a178a2b3f4f594f3dd370c4">459f75e7</a>. At last we have quite proper progress evaluation and visualization that is extremely valuable for large production jobs.</p>