Klever

Evgeny Novikov wrote:

I am wondering, why the suggested interface differs so much from the one we discussed within parent issue #8149. In particular, progress requests will be sent by Core that most likely means that job identifiers aren't needed. I guess that we don't need statistics on the basis of all jobs while there are many other information to be sent, say, the number of solved/failed tasks, a sub-jobs decision progress, etc.

Ilja said that this feature will work in the way I've done. Then I understood it icorrectly and need documentation of this feature.

I meant that a new implementation in Bridge would be just simpler than existing and not many things should be done there (from my perspective).

If you need a bit more detailed documentation, then it is better to wait until I finish the implementation in Core because there are a lot of changes here should be done and tricky issues to be solved.

All works quite fine. I merged the branch to 8149-new-progress (with my implementation in Core).

The only note about guards in bridge.service.utils. It assumes that process is present but sometimes for failed jobs it is not so, so I slightly modified two checks. Please, review the fix (the last commit in branch 8149-new-progress).

Perhaps some issues will gone after reloading JavaScripts, but I am not sure.

Ilja Zakharov wrote:

In general it works but there are a couple of issues left:
- Still progress is hidden for solved jobs (even if they had it calculated) but in the last version of the document I have added this requirement (see the update comments which I added in the morning).

It should work. Updating cached javascript will not help as dependency of job status is in python script. Maybe you had wrong version or the server is not reloaded after version change.

- I am not completely sure, but once I saw that if progress is kept on the main page (for a failed job) then after starting solving it fields with tasks and subjobs solution dates left with previous values from the last run and not cleaned.

It can't be so as for all starting jobs function __create_solving_progress() is called (service/utils.py: 892). SolvingProgress is created for all started jobs and will be deleted in this function together with JobProgress (where progress dates are) before job start.

- I also encountered several other problems but I had to fix them to try running jobs. So please review my changes I made in the last commit.

Reviewed.

It should work. Updating cached javascript will not help as dependency of job status is in python script. Maybe you had wrong version or the server is not reloaded after version change.

I tried this on my machine after restarting Bridge and browser and in a virtual machine with a completely new installation. If the job is successfully solved then Bridge hides all progress information. Also Bridge hides progress information for failed jobs if the progress is 100%.

Also you reverted one of my fixes, but without it I see the exception sometimes:

Traceback (most recent call last):
File "/var/www/klever-bridge/jobs/views.py", line 313, in get_job_data
progress = GetJobsProgresses(request.user, [job.id]).data[job.id]
File "/var/www/klever-bridge/service/utils.py", line 1001, in init
self.data = self.__get_data(jobs_ids)
File "/var/www/klever-bridge/service/utils.py", line 1026, in _get_data
data[j_id] = self._job_values(j_id)
File "/var/www/klever-bridge/service/utils.py", line 1067, in __job_values
elif self._j_progress[j_id].gag_text_ts is not None:
KeyError: 5

Ilja Zakharov wrote:

It should work. Updating cached javascript will not help as dependency of job status is in python script. Maybe you had wrong version or the server is not reloaded after version change.

I tried this on my machine after restarting Bridge and browser and in a virtual machine with a completely new installation. If the job is successfully solved then Bridge hides all progress information. Also Bridge hides progress information for failed jobs if the progress is 100%.

Could you download and send me such job?

Also you reverted one of my fixes, but without it I see the exception sometimes:

Sorry, I thought it can't be because of line 1030.

Could you download and send me such job?

Send it by an e-mail.

Sorry, I thought it can't be because of line 1030.

Seems that in particular, it happens when the status is Pending or processing and a new progress status is not received yet.

Ilja Zakharov wrote:

Could you download and send me such job?

Send it by an e-mail.

The reason is "Expected time for solving subjobs" and "Expected time for solving tasks" both are null. It should be 0 after a job is solved or another number if it is failed.

Hmm, maybe because I send textual value at the end of solution? This is the example of data with progress:

{'start tasks solution': True}
{'expected time for solving tasks': 60, 'solved tasks': 3, 'failed tasks': 0, 'total tasks to be generated': 9}
{'solved tasks': 3, 'failed tasks': 0, 'expected time for solving tasks': 50}
{'solved tasks': 3, 'failed tasks': 0, 'expected time for solving tasks': 38}
{'solved tasks': 8, 'failed tasks': 0, 'expected time for solving tasks': 8}
{'finish tasks solution': True, 'solved tasks': 9, 'failed tasks': 0, 'expected time for solving tasks': 'Solution finished'}

Ilja Zakharov wrote:

Hmm, maybe because I send textual value at the end of solution? This is the example of data with progress:
[...]

Yes, it is the reason. Expected time is shown only for processing jobs. In other cases it is not shown and don't need the textual value. The progress can be shown if it can be calculated and expected time is known (like it were described in docs). The second condition is false for textual values.