Project

General

Profile

Feature #8729

Bridge does not work with Django 2.*

Added by Ilja Zakharov about 1 year ago. Updated 9 months ago.

Status:
Closed
Priority:
Urgent
Category:
Bridge
Target version:
Start date:
02/19/2018
Due date:
% Done:

100%

Estimated time:
Published in build:

Description

After Django update from 1st to 2nd version several optional parameters become necessary. To support the new release of Django it is necessary now to provide several arguments explicitly.


Related issues

Related to Klever - Bug #8905: Take into account that Django development server is not powerfulClosed2018-05-30

Related to Klever - Bug #8934: Incorrect processing of decision configuration fileClosed2018-06-08

Related to Klever - Bug #9027: Fix Bridge unit testsClosed2018-06-28

History

#1 Updated by Vladimir Gratinskiy 9 months ago

  • % Done changed from 0 to 100

Implemented in "new-report-data".

#2 Updated by Vladimir Gratinskiy 9 months ago

  • Status changed from New to Feedback

#3 Updated by Evgeny Novikov 9 months ago

  • Target version set to 1.0
  • Priority changed from High to Urgent

Raise the issue priority since its implementation was performed together with high priority issues.

#4 Updated by Evgeny Novikov 9 months ago

  • Status changed from Feedback to Resolved

Works for me.

#5 Updated by Ilja Zakharov 9 months ago

  • Status changed from Resolved to Open

Cannot change mode (production or development) on the page of job solution. After switching I see an empty page.

#6 Updated by Ilja Zakharov 9 months ago

There is an another but more important bug. Cannot upload solved job obtained in the same new version of Bridge:

Uploading reports failed: null value in column "source_id" violates not-null constraint
DETAIL: Failing row contains (1375, Unsafes/2018/05/ErrorTrace_qA1rUtn.zip, 5, 257560576, 6746, 3615, f, , null).
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
psycopg2.IntegrityError: null value in column "source_id" violates not-null constraint
DETAIL: Failing row contains (1375, Unsafes/2018/05/ErrorTrace_qA1rUtn.zip, 5, 257560576, 6746, 3615, f, , null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 705, in create_job_from_tar
resources, coverage_data, coverage_files, attr_data)
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 773, in _init

self.
_upload_all()
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 819, in upload_all
self.__upload_unsafe_reports()
File "/usr/lib/python3.6/contextlib.py", line 52, in inner
return func(*args, **kwds)
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 886, in _upload_unsafe_reports
report.add_trace(REPORT_ARCHIVE['error trace'], fp, True)
File "/work/zakharov/src/klever/bridge/reports/models.py", line 215, in add_trace
self.error_trace.save(fname, File(fp), save)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/fields/files.py", line 93, in save
self.instance.save()
File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py", line 729, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py", line 759, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py", line 842, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/base.py", line 880, in _do_insert
using=using, raw=raw)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 1125, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py", line 1283, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/work/zakharov/src/klever/bridge/bridge/
_init
.py", line 39, in execute_wrapper
return original(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 77, in execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py", line 89, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "source_id" violates not-null constraint
DETAIL: Failing row contains (1375, Unsafes/2018/05/ErrorTrace_qA1rUtn.zip, 5, 257560576, 6746, 3615, f, , null).
Stack (most recent call last):
File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/socketserver.py", line 639, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.6/socketserver.py", line 696, in init
self.handle()
File "/usr/local/lib/python3.6/dist-packages/django/core/servers/basehttp.py", line 154, in handle
handler.run(self.server.get_app())
File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/usr/local/lib/python3.6/dist-packages/django/contrib/staticfiles/handlers.py", line 66, in call
return self.application(environ, start_response)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/wsgi.py", line 146, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py", line 81, in get_response
response = self._middleware_chain(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py", line 95, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/work/zakharov/src/klever/bridge/bridge/utils.py", line 265, in call
response = self.get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py", line 126, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py", line 69, in view
return self.dispatch(request, *args, **kwargs)
File "/work/zakharov/src/klever/bridge/tools/profiling.py", line 170, in dispatch
response = getattr(super(), 'dispatch')(request, *args, **kwargs)
File "/work/zakharov/src/klever/bridge/bridge/CustomViews.py", line 59, in dispatch
return getattr(super(), 'dispatch')(request, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py", line 89, in dispatch
return handler(request, *args, **kwargs)
File "/work/zakharov/src/klever/bridge/bridge/CustomViews.py", line 92, in post
return JsonResponse(self.get_context_data())
File "/work/zakharov/src/klever/bridge/jobs/views.py", line 360, in get_context_data
UploadJob(parent, self.request.user, job_dir.name)
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 532, in init
self.
_create_job_from_tar()
File "/work/zakharov/src/klever/bridge/jobs/Download.py", line 710, in __create_job_from_tar
logger.exception("Uploading reports failed: %s" % e, stack_info=True)
Deleting ReportRoot files took 0.01015329360961914 seconds.

#7 Updated by Evgeny Novikov 9 months ago

  • Related to Bug #8905: Take into account that Django development server is not powerful added

#8 Updated by Ilja Zakharov 9 months ago

Another one issue: trying to delete a newly created mark I got:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/detail.py", line 51, in get_object
obj = queryset.get()
File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 403, in get
self.model._meta.object_name
reports.models.DoesNotExist: ReportUnsafe matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py", line 126, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py", line 69, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/utils/decorators.py", line 62, in _wrapper
return bound_func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/utils/decorators.py", line 58, in bound_func
return func.
_get__(self, type(self))(*args2, **kwargs2)
File "/work/zakharov/src/klever/bridge/tools/profiling.py", line 170, in dispatch
response = getattr(super(), 'dispatch')(request, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/base.py", line 89, in dispatch
return handler(request, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/detail.py", line 105, in get
self.object = self.get_object()
File "/usr/local/lib/python3.6/dist-packages/django/views/generic/detail.py", line 54, in get_object {'verbose_name': queryset.model._meta.verbose_name})
django.http.response.Http404: No report unsafe found matching the query

#9 Updated by Vladimir Gratinskiy 9 months ago

  • Status changed from Open to Feedback

Exceptions were fixed. Uploading jobs should work now.

#10 Updated by Ilja Zakharov 9 months ago

Yes, works nicely. The only issue is that loading of old archives fails with "The job archive was corrupted" (because you changed tag "safe marks") but it is better to see correct reason "The job archive format is not supported".

#11 Updated by Evgeny Novikov 9 months ago

  • Status changed from Feedback to Resolved

Ilja Zakharov wrote:

Yes, works nicely. The only issue is that loading of old archives fails with "The job archive was corrupted" (because you changed tag "safe marks") but it is better to see correct reason "The job archive format is not supported".

Let's do it separately, since it isn't related issue indeed (#8926).

#12 Updated by Ilja Zakharov 9 months ago

Minor bug: even when job is solved when I click at "solved" button and observe component reports in the top left corner status is "In progress" which is incorrect.

#13 Updated by Vladimir Gratinskiy 9 months ago

Ilja Zakharov wrote:

Minor bug: even when job is solved when I click at "solved" button and observe component reports in the top left corner status is "In progress" which is incorrect.

Fixed. It was just a visualisation bug.

#14 Updated by Evgeny Novikov 9 months ago

  • Status changed from Open to Resolved

It works.

#15 Updated by Evgeny Novikov 9 months ago

  • Status changed from Resolved to Closed

I merged the branch to master in 4339da97 as a part of branch core-new-formats.

For migrating old databases one needs to do one more extra step:

./manage.py migrate --fake users 0001_initial
./manage.py migrate --fake jobs 0001_initial
./manage.py migrate --fake reports 0001_initial
./manage.py migrate --fake marks 0001_initial
./manage.py migrate --fake service 0001_initial
./manage.py migrate --fake tools 0001_initial

Be careful! I do recommend to backup data first, if you really treat it important from some point of view.

#16 Updated by Evgeny Novikov 9 months ago

  • Related to Bug #8934: Incorrect processing of decision configuration file added

#17 Updated by Evgeny Novikov 8 months ago

  • Related to Bug #9027: Fix Bridge unit tests added

Also available in: Atom PDF