Klever

The reason is the job has unfinished reports. But when you upload finish Core report there is check, if there are unfinished report, the job will become "Corrupted". Without this report the job can't become "Failed" or "Finished" (you can download only such jobs). So you shouldn't do what you do with unfinished jobs.

Evgeny Novikov wrote:

I didn't noticed any changes. I still can download archives of jobs having unfinished reports. Their uploading still results in exceptions like in the issue description.

But you can't make new Failed jobs with unfinished reports anymore.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I didn't noticed any changes. I still can download archives of jobs having unfinished reports. Their uploading still results in exceptions like in the issue description.

But you can't make new Failed jobs with unfinished reports anymore.

I attached the example of such the job recently ;)

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I didn't noticed any changes. I still can download archives of jobs having unfinished reports. Their uploading still results in exceptions like in the issue description.

But you can't make new Failed jobs with unfinished reports anymore.

I attached the example of such the job recently ;)

I mean you can't decide the job so its status will be Failed and there will be unfinished reports. Old jobs are not affected by this fix.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

I didn't noticed any changes. I still can download archives of jobs having unfinished reports. Their uploading still results in exceptions like in the issue description.

But you can't make new Failed jobs with unfinished reports anymore.

I attached the example of such the job recently ;)

I mean you can't decide the job so its status will be Failed and there will be unfinished reports. Old jobs are not affected by this fix.

I decide that job after checkouting your branch. So I did get this job archive although I couldn't do that.

Fixed after merge to branch "feature_7902". Now if:
1) scheduler says the job is failed it will be FAILED;
2) scheduler says the job is solved:
- if the job has unfinished reports then the job is CORRUPTED;
- elif the job is full-weight and there are unknown reports for Core component then the job is FAILED;
- else the job is SOLVED.

There are no other ways when the job will become SOLVED.

Vladimir Gratinskiy wrote:

Fixed after merge to branch "feature_7902". Now if:
1) scheduler says the job is failed it will be FAILED;
2) scheduler says the job is solved:
- if the job has unfinished reports then the job is CORRUPTED;
- elif the job is full-weight and there are unknown reports for Core component then the job is FAILED;
- else the job is SOLVED.

There are no other ways when the job will become SOLVED.

But if the job has status FAILED due to the scheduler failed it but it has unfinished reports, it can be downloaded while its uploading will fail, will it?

Evgeny Novikov wrote:

But if the job has status FAILED due to the scheduler failed it but it has unfinished reports, it can be downloaded while its uploading will fail, will it?

Exactly.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

But if the job has status FAILED due to the scheduler failed it but it has unfinished reports, it can be downloaded while its uploading will fail, will it?

Exactly.

So, I don't understand why Bridge doesn't mark corresponding jobs as corrupted. The only reason for that is to show an error message from the scheduler that is associated with status FAILED. But it would be better to associate it with an error message for CORRUPTED in addition to its own error message.

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

But if the job has status FAILED due to the scheduler failed it but it has unfinished reports, it can be downloaded while its uploading will fail, will it?

Exactly.

So, I don't understand why Bridge doesn't mark corresponding jobs as corrupted. The only reason for that is to show an error message from the scheduler that is associated with status FAILED. But it would be better to associate it with an error message for CORRUPTED in addition to its own error message.

Oh, sorry. Uploading will not fail. I will open downloading/uploading all jobs (except pending and processing). I didn't find any problems with uploading such jobs now, but we will find it when try to use it. The branch feature_7902 is ready for testing.

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

But if the job has status FAILED due to the scheduler failed it but it has unfinished reports, it can be downloaded while its uploading will fail, will it?

Exactly.

So, I don't understand why Bridge doesn't mark corresponding jobs as corrupted. The only reason for that is to show an error message from the scheduler that is associated with status FAILED. But it would be better to associate it with an error message for CORRUPTED in addition to its own error message.

Oh, sorry. Uploading will not fail. I will open downloading/uploading all jobs (except pending and processing). I didn't find any problems with uploading such jobs now, but we will find it when try to use it. The branch feature_7902 is ready for testing.

But it looks like some inconsistency.

Status Corrupted is intended for jobs which verification results are malformed, e.g. this is the case when Core or its components sends unexpected reports or a reports tree is broken after all.

Status Canceled is very close to Corrupted but in this case the user stops a job decision and most likely verification results are malformed.

Status Failed is intended for jobs which verification results are well-formed but something bad happens but Core and its components could detect and report this.

Status Solved is intended for jobs with good verification results.

Unfortunately there is no place where the scheduler can report errors detected on top of a job decision, e.g. information about fatal errors and time/memory outs. So these errors are specified as popups for status Failed even though corresponding jobs verification results are malformed.

At first I would like to understand whether this is correct and complete description.

Evgeny Novikov wrote:

Unfortunately there is no place where the scheduler can report errors detected on top of a job decision, e.g. information about fatal errors and time/memory outs. So these errors are specified as popups for status Failed even though corresponding jobs verification results are malformed.

Popups contains error either from scheduler or from bridge (any exception description). For example, if report has wrong format, uploading report would raise the exception that will be caught and saved to SolvingProgress.error field.

Status Solved is for good verification results (that were uploaded without any problems) and with full set of reports (all reports are finished).
Status Failed is for good verification results but scheduler said that the job is Failed. This status is also for jobs with good verification results with unknown report for Core.
Status Corrupted is for other cases. For example, the scheduler was disconnected, or uploading report failed, or scheduler think everything is OK and status should be Solved but bridge found unfinished reports.
The difference of Cancelled status from Corrupted is the first one just shows that user corrupted results, not service.

So, I suggest to introduce one more status to specify errors related with schedulers, that are both errors it explicitly reports (e.g. that a job decision was terminated due to timeout) and other errors like disconnecting. Let's call this status Terminated since it corresponds to the most of such errors.

Then we will not have ambiguity for statuses Failed and Corrupted.

Evgeny Novikov wrote:

So, I suggest to introduce one more status to specify errors related with schedulers, that are both errors it explicitly reports (e.g. that a job decision was terminated due to timeout) and other errors like disconnecting. Let's call this status Terminated since it corresponds to the most of such errors.

Then we will not have ambiguity for statuses Failed and Corrupted.

When this status should be set? When scheduler incorrectly set the status of the job to Solved while there are unfinished reports or Core have unknown report (for full-weight jobs)?

Vladimir Gratinskiy wrote:

Evgeny Novikov wrote:

So, I suggest to introduce one more status to specify errors related with schedulers, that are both errors it explicitly reports (e.g. that a job decision was terminated due to timeout) and other errors like disconnecting. Let's call this status Terminated since it corresponds to the most of such errors.

Then we will not have ambiguity for statuses Failed and Corrupted.

When this status should be set? When scheduler incorrectly set the status of the job to Solved while there are unfinished reports or Core have unknown report (for full-weight jobs)?

When schedulers report errors themselves (you described this as "Status Failed is for good verification results but scheduler said that the job is Failed." - actually results can be malformed, e.g. there can be unfinished reports) or they are disconnected, or there are some other issues related with schedulers.

Fixed. Now Failed status just for jobs with unknown report for Core.

The status of jobs with unfinished reports if scheduler didn't report an error will be Corrupted.