Project

General

Profile

Bug #9151

Bridge starts job even if job.json or tasks.json is invalid

Added by Ilja Zakharov 10 months ago. Updated 8 months ago.

Status:
Closed
Priority:
Urgent
Category:
Bridge
Target version:
Start date:
07/23/2018
Due date:
% Done:

0%

Estimated time:
Detected in build:
svn
Platform:
Published in build:

Description

If job.json/tasks.json cannot be parsed Bridge anyway sets job status as pending and sends it to the scheduler. Then either the scheduler fails because it can not get task resource limitations or Core fails at parsing the job description.

The solution should be that Bridge forbids solving job if its job.json/tasks.json cannot be parsed.


Related issues

Related to Klever - Feature #8937: Highlight JSON files syntax and check its correctness in the online editorClosed06/09/2018

Actions
Related to Klever - Bug #9289: Native Scheduler does not properly handle errors with tasks.jsonClosed09/28/2018

Actions

History

#1

Updated by Evgeny Novikov 9 months ago

  • Related to Feature #8937: Highlight JSON files syntax and check its correctness in the online editor added
#2

Updated by Vladimir Gratinskiy 8 months ago

  • Status changed from New to Resolved

Fixed in bridge-2.0

#3

Updated by Evgeny Novikov 8 months ago

  • Status changed from Resolved to Open
  • Description updated (diff)
  • Subject changed from Bridge starts job even if tasks.json is invalid to Bridge starts job even if job.json or tasks.json is invalid

Although, thanks to #8937 I can not break JSON files through editing them, I can replace JSON files with ones having an invalid format. Then Bridge allows to decide corresponding jobs with raising the following exception:

Expecting property name enclosed in double quotes: line 8 column 1 (char 152)
Traceback (most recent call last):
  File "/home/novikov/work/klever/bridge/service/utils.py", line 544, in __get_tasks_limits
    return json.load(fp)
  File "/usr/lib64/python3.6/json/__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 8 column 1 (char 152)

After all schedulers fail as expected.

BTW, there are many various files in jobs. But just 2 of them are vital, job.json and tasks.json. So, I extended the issue from tasks.json to job.json as well.

#4

Updated by Evgeny Novikov 8 months ago

  • Related to Bug #9289: Native Scheduler does not properly handle errors with tasks.json added
#5

Updated by Vladimir Gratinskiy 8 months ago

  • Status changed from Open to Resolved

Fixed in bridge-2.0 for all *.json files.

#6

Updated by Evgeny Novikov 8 months ago

  • Status changed from Resolved to Open

Please, make following changes in branch bridge-2.0-master!

I revealed that Python 3.5 can't handle in jobs.jobForm.UploadFilesTree#__check_jsons:

            with f.file.file as fp:
                try:
                    json.load(fp)

There is such the error: "the JSON object must be str, not 'bytes'".

Can you fix that?

#7

Updated by Vladimir Gratinskiy 8 months ago

  • Status changed from Open to Resolved

Evgeny Novikov wrote:

Can you fix that?

Fixed.

#8

Updated by Evgeny Novikov 8 months ago

Thank you! I renamed the branch back to bridge-2.0 while former bridge-2.0 to bridge-2.0-with-klever-2.0. At the moment bridge-2.0 undergoes testing. In case of success I will be able to merge it directly to master unless klever-2.0 and bridge-2.0-with-klever-2.0 that aren't completed still and introduces numerous changes in testing/validation jobs and marks.

#9

Updated by Evgeny Novikov 8 months ago

  • Status changed from Resolved to Closed

Tests passed, so, I merged the branch to master in b8bda00ac.

Also available in: Atom PDF