https://forge.ispras.ru/https://forge.ispras.ru/favicon.ico?16490126692019-01-23T10:13:19ZOpen-Source ProjectsKlever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=355122019-01-23T10:13:19ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-5 priority-4 priority-default closed parent behind-schedule" href="/issues/9414">Feature #9414</a>: Reuse archive with source files between different error traces on report uploading</i> added</li></ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=355532019-01-25T12:29:19ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Subject</strong> changed from <i>Support hashing build bases to avoid copying them each time at OpenStack deployment</i> to <i>Do not deploy the same build bases</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/35553/diff?detail_id=40265">diff</a>)</li></ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=356432019-02-25T08:13:25ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Assignee</strong> set to <i>Evgeny Novikov</i></li></ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=358602019-03-24T12:03:36ZEvgeny Novikovnovikov@ispras.ru
<ul></ul><p>With <a class="external" href="https://github.com/17451k/clade/issues/41">https://github.com/17451k/clade/issues/41</a> we can distinguish build bases by their unique identifier.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=365452019-07-15T08:56:44ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>High</i></li><li><strong>Target version</strong> deleted (<del><i>3.0</i></del>)</li></ul><p>We have no time to implement these tasks in Klever <a class="version" href="https://forge.ispras.ru/versions/264">3.0</a>.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=380392020-02-05T13:09:23ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Blocks</strong> <i><a class="issue tracker-4 status-1 priority-5 priority-high3" href="/issues/10101">Feature #10101</a>: Deploy addons and build bases within base images</i> added</li></ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=382612020-03-02T11:32:04ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Has duplicate</strong> <i><a class="issue tracker-1 status-6 priority-6 priority-high2 closed" href="/issues/9696">Bug #9696</a>: Klever deployment script removes build bases</i> added</li></ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=393632020-07-13T11:40:45ZIlya Shchepetkovshchepetkov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li><li><strong>Assignee</strong> changed from <i>Evgeny Novikov</i> to <i>Ilya Shchepetkov</i></li></ul><p>Implemented in the improve-openstack-deployment branch using rsync. Now Klever sources, addons and build bases are deployed only if there are some changes in them. As a consequence, deployment speed during "update" action is considerably faster then before.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=394772020-08-13T13:07:39ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Urgent</i></li><li><strong>Target version</strong> set to <i>3.0</i></li></ul><p>Let's include this into <a class="version" href="https://forge.ispras.ru/versions/264">3.0</a>.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=394792020-08-13T14:02:50ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Open</i></li><li><strong>Assignee</strong> deleted (<del><i>Ilya Shchepetkov</i></del>)</li><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>High</i></li><li><strong>Target version</strong> deleted (<del><i>3.0</i></del>)</li></ul><p>Branch <em>improve-openstack-deployment</em> avoids copying the same local build bases to instances, but it does not solve the problem in general. In general we need some means like for addons which are not updated (nothing is downloaded, extracted and so on) until their versions change within a deployment configuration file.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395492020-09-04T18:08:45ZIlya Shchepetkovshchepetkov@ispras.ru
<ul><li><strong>Assignee</strong> set to <i>Ilya Shchepetkov</i></li></ul><p>Implemented in the deploy-build-bases branch, which also contains other improvements regarding deployment of build bases:</p>
<ul>
<li>Now deployment scripts automatically extract build bases during deployment (previously they were simply copied)</li>
<li>Now build bases are deployed during 'update' action only if they are changed</li>
<li>Now deployment of build bases and Klever Addons uses the same code, which should simplify further improvements</li>
</ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395502020-09-07T15:53:46ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Urgent</i></li><li><strong>Target version</strong> set to <i>3.0</i></li></ul><p>Since it is solved, let's include it within Klever <a class="version" href="https://forge.ispras.ru/versions/264">3.0</a>.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395512020-09-07T15:54:20ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Resolved</i></li></ul><p>Likely resolved. Needs testing.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395522020-09-07T16:47:38ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Open</i></li></ul><p>It works, but due to ill-formed archives (e.g. <a class="external" href="https://forge.ispras.ru/attachments/7328">https://forge.ispras.ru/attachments/7328</a>) users may meet very strange issues because of different build bases can be intermixed in the unpredictable way within the same directory.</p>
I suggest to check before installation to the final destination the following:
<ol>
<li>A build base is placed within at least one top-level directory within an archive.</li>
<li>There is not most low-level directory that contains the build base at the deployment directory already.</li>
</ol>
<p>Perhaps, some assistance from Clade will be necessary, since just Clade knows what is a build base.</p> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395892020-09-17T18:34:54ZIlya Shchepetkovshchepetkov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Resolved</i></li></ul><p>I have added the following changes to the same deploy-build-bases branch:</p>
<ul>
<li>Now data transferred by rsync is compressed (unless the data itself is an archive)</li>
<li>Now paths with spaces are handled correctly by rsync</li>
<li>Now there is an optional ability to manually specify versions of build bases (but by default versions are calculated automatically)</li>
<li>Now each build base has a unique name specified in the klever.json file. This name can be used in the job.json file to refer to the required build base.</li>
</ul> Klever - Feature #9356: Do not deploy the same build baseshttps://forge.ispras.ru/issues/9356?journal_id=395912020-09-18T15:46:15ZEvgeny Novikovnovikov@ispras.ru
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li></ul><p>We tested and fixed some additional related issues, after all everything seems to work perfect. I merged the branch to master in <a class="changeset" title="Put build bases to directory "build bases" in klever deployment directory This breaks backward c..." href="https://forge.ispras.ru/projects/klever/repository/331/revisions/4a52fad75a0c87fdb57c99cabdfab0c22345fe87">4a52fad75</a>. Everybody is welcome to deploy build bases within their Klever instances!</p>
Important remark. Existent Klever instances deployed by means of scripts need some manual migrations, e.g. OpenStack needs the following (I hope that local instances can be easily reinstalled or you may try steps like below):
<ol>
<li>Edit <em>/etc/default/klever</em>. Change <em>KLEVER_DATA_DIR="/home/debian/klever-inst/klever/build bases"</em> to <em>KLEVER_DATA_DIR="/home/debian/klever-inst/build bases"</em>.</li>
<li>Move directory <em>/home/debian/klever-inst/klever/build bases</em> to <em>/home/debian/klever-inst/build bases</em>.</li>
<li>Restart the scheduler: "sudo systemctl restart klever-native-scheduler.service".</li>
</ol>