Feature #9356
closedDo not deploy the same build bases
0%
Description
Deployment scripts in the current version copy build bases each time. It is better to compare hashes for build bases to speedup update procedure.
Updated by Evgeny Novikov almost 6 years ago
- Related to Feature #9414: Reuse archive with source files between different error traces on report uploading added
Updated by Evgeny Novikov almost 6 years ago
- Subject changed from Support hashing build bases to avoid copying them each time at OpenStack deployment to Do not deploy the same build bases
- Description updated (diff)
Updated by Evgeny Novikov over 5 years ago
With https://github.com/17451k/clade/issues/41 we can distinguish build bases by their unique identifier.
Updated by Evgeny Novikov over 5 years ago
- Priority changed from Urgent to High
- Target version deleted (
3.0)
We have no time to implement these tasks in Klever 3.0.
Updated by Evgeny Novikov almost 5 years ago
- Blocks Feature #10101: Deploy addons and build bases within base images added
Updated by Evgeny Novikov over 4 years ago
- Has duplicate Bug #9696: Klever deployment script removes build bases added
Updated by Ilya Shchepetkov over 4 years ago
- Status changed from New to Resolved
- Assignee changed from Evgeny Novikov to Ilya Shchepetkov
Implemented in the improve-openstack-deployment branch using rsync. Now Klever sources, addons and build bases are deployed only if there are some changes in them. As a consequence, deployment speed during "update" action is considerably faster then before.
Updated by Evgeny Novikov over 4 years ago
- Priority changed from High to Urgent
- Target version set to 3.0
Let's include this into 3.0.
Updated by Evgeny Novikov over 4 years ago
- Status changed from Resolved to Open
- Assignee deleted (
Ilya Shchepetkov) - Priority changed from Urgent to High
- Target version deleted (
3.0)
Branch improve-openstack-deployment avoids copying the same local build bases to instances, but it does not solve the problem in general. In general we need some means like for addons which are not updated (nothing is downloaded, extracted and so on) until their versions change within a deployment configuration file.
Updated by Ilya Shchepetkov about 4 years ago
- Assignee set to Ilya Shchepetkov
Implemented in the deploy-build-bases branch, which also contains other improvements regarding deployment of build bases:
- Now deployment scripts automatically extract build bases during deployment (previously they were simply copied)
- Now build bases are deployed during 'update' action only if they are changed
- Now deployment of build bases and Klever Addons uses the same code, which should simplify further improvements
Updated by Evgeny Novikov about 4 years ago
- Priority changed from High to Urgent
- Target version set to 3.0
Since it is solved, let's include it within Klever 3.0.
Updated by Evgeny Novikov about 4 years ago
- Status changed from Open to Resolved
Likely resolved. Needs testing.
Updated by Evgeny Novikov about 4 years ago
- Status changed from Resolved to Open
It works, but due to ill-formed archives (e.g. https://forge.ispras.ru/attachments/7328) users may meet very strange issues because of different build bases can be intermixed in the unpredictable way within the same directory.
I suggest to check before installation to the final destination the following:- A build base is placed within at least one top-level directory within an archive.
- There is not most low-level directory that contains the build base at the deployment directory already.
Perhaps, some assistance from Clade will be necessary, since just Clade knows what is a build base.
Updated by Ilya Shchepetkov about 4 years ago
- Status changed from Open to Resolved
I have added the following changes to the same deploy-build-bases branch:
- Now data transferred by rsync is compressed (unless the data itself is an archive)
- Now paths with spaces are handled correctly by rsync
- Now there is an optional ability to manually specify versions of build bases (but by default versions are calculated automatically)
- Now each build base has a unique name specified in the klever.json file. This name can be used in the job.json file to refer to the required build base.
Updated by Evgeny Novikov about 4 years ago
- Status changed from Resolved to Closed
We tested and fixed some additional related issues, after all everything seems to work perfect. I merged the branch to master in 4a52fad75. Everybody is welcome to deploy build bases within their Klever instances!
Important remark. Existent Klever instances deployed by means of scripts need some manual migrations, e.g. OpenStack needs the following (I hope that local instances can be easily reinstalled or you may try steps like below):- Edit /etc/default/klever. Change KLEVER_DATA_DIR="/home/debian/klever-inst/klever/build bases" to KLEVER_DATA_DIR="/home/debian/klever-inst/build bases".
- Move directory /home/debian/klever-inst/klever/build bases to /home/debian/klever-inst/build bases.
- Restart the scheduler: "sudo systemctl restart klever-native-scheduler.service".