Klever

Alexey Khoroshilov observed several times a strange issue when at several instances during several minutes request get_jobs_and_tasks failed with the following exception:

2018-01-12 21:07:39,240 (__init__.py:484) root ERROR> An error occured:
Traceback (most recent call last):
  File "/home/debian/klever/scheduler/bin/../schedulers/__init__.py", line 180, in launch
    server_state = self.server.exchange(scheduler_state)
  File "/home/debian/klever/scheduler/bin/../server/bridge.py", line 50, in exchange
    ret = self.session.json_exchange("service/get_jobs_and_tasks/", data)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 168, in json_exchange
    response = self.__request(endpoint, 'POST', json_data, looping=looping)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 95, in __request
    'Got error "{0}" when send "{1}" request to "{2}"'.format(self.error, method, url))
utils.bridge.BridgeError: Got error "You are not signing in" when send "POST" request to "http://localhost:8998/service/get_jobs_and_tasks/" 
2018-01-12 21:07:39,255 (__init__.py:131) root ERROR> Uncaught exception
Traceback (most recent call last):
  File "/home/debian/klever/scheduler/bin/../schedulers/__init__.py", line 180, in launch
    server_state = self.server.exchange(scheduler_state)
  File "/home/debian/klever/scheduler/bin/../server/bridge.py", line 50, in exchange
    ret = self.session.json_exchange("service/get_jobs_and_tasks/", data)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 168, in json_exchange
    response = self.__request(endpoint, 'POST', json_data, looping=looping)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 95, in __request
    'Got error "{0}" when send "{1}" request to "{2}"'.format(self.error, method, url))
utils.bridge.BridgeError: Got error "You are not signing in" when send "POST" request to "http://localhost:8998/service/get_jobs_and_tasks/" 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/debian/klever/scheduler/bin/native-scheduler", line 38, in <module>
    scheduler_impl.launch()
  File "/home/debian/klever/scheduler/bin/../schedulers/__init__.py", line 485, in launch
    self.terminate()
  File "/home/debian/klever/scheduler/bin/../schedulers/native.py", line 283, in terminate
    self.server.submit_nodes(configurations, looping=False)
  File "/home/debian/klever/scheduler/bin/../server/bridge.py", line 86, in submit_nodes
    self.session.json_exchange("service/update_nodes/", data, looping=looping)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 168, in json_exchange
    response = self.__request(endpoint, 'POST', json_data, looping=looping)
  File "/home/debian/klever/scheduler/bin/../utils/bridge.py", line 95, in __request
    'Got error "{0}" when send "{1}" request to "{2}"'.format(self.error, method, url))
utils.bridge.BridgeError: Got error "You are not signing in" when send "POST" request to "http://localhost:8998/service/update_nodes/" 

I didn't find out causes of this issue. So, I suppose to implement a trivial workaround like with bad zip archives - to log the issue and to re-authenticate Native Scheduler and its workers.

Perhaps Core and its components also suffer from this issue. So, there is some sense to implement the same workaround for them in advance as well.