2022-09-12 Registry migration plan

Owned by Fritz Lehnert
Last updated: Sept 12, 2022 by Maximilian Huber
2 min read

Attendees:

  • @Timothee Dzik 

  • @Maximilian Huber 

  • @Nils Semmelrock 

  • @Jan Heidbrink 

  • @Fritz Lehnert 

  • Fabio Palumbo (LF)

New Linux Foundation Artifactory

https://magma.linuxfoundation.jfrog.org/

Current status

  • As you all know we currently use https://artifactory.magmacore.org/ to host all our packages from debian to docker images (blob, helm etc ).

  • Documentation is here: https://lf-magma.atlassian.net/wiki/display/HOME/DevOps+@Magma (11. registry )

  • The JFrog license that we use expires on 3nd December 2022 00:00:00 and the LF / board decided not to renew it since we’re tending to only use Linux Foundation products.

  • We have to start the migration as soon as possible and for that we need to split the tasks across all DevOps team.

  • Artifactory is hosted on OVH. Traffic costs are currently unclear?

    • @Jan Heidbrink Who can access the OVH infrastructure?

The plan

Phase 1: Copy all production packages and change upload CI of existing workflows to the new registry

  • Ideas of @Timothee Dzik :

    • Timeline: End of September 2022

    • Copy all production packages: Fabio Palumbo would be best to do that. He has full / best access to that new registry.

    • Changes to CI could be done by @Timothee Dzik 

  • @Nils Semmelrock Question: LF said this would only be possible by CI. Do we need a CI for that than?

    • Fabio should do that. He has the necessary permissions.

  • @Maximilian Huber If we migrate v1.6 and v1.7 to the new Artifactory, how do we test this worked?

    • @Timothee Dzik Two possible ways: Fresh install and Upgrade.

Phase 2: Write missing pipelines for the dependencies 

  • Ideas of @Timothee Dzik :

    • Timeline: Mid of November 2022

    • Support from TNG is highly appreciated.

  • @Nils Semmelrock Question: Can we delete artifacts from Artifactory?

    • Deletion was told to be only possible via support ticket to LF.

  • @Maximilian Huber : The Bazel migration does reduce the number of dependencies is largely reduced. This might be a race condition for phase 2.

    • TNG team is currently working on Bazel migration.

  • @Maximilian Huber : While working on OVS, there are changes to the source code management planned. This should be already implemented or respected with the new build scheme.

Phase 3: Proxy the old registry to the new one and do all the necessary code changes 

  • Ideas of @Timothee Dzik :

    • Timeline: End of November 2022

    • If we can proxy the traffic from the old to the new registry, this would be seen as ideal.

  • @Jan Heidbrink Concerned that not doing a code change would not be the cleanest solution. This code change is very well do-able and would be better going forward.

    • @Maximilian Huber Agrees. A proxy could still be setup for some of the existing users.