Release Testing

Owned by Maximilian Huber
Last updated: Aug 18, 2022
2 min read

Contact persons:

for Wavelabs: @Yogesh Pandey 

for Radtonics:  @Sujay 

for Sercomm: @Xiaoan Chen 

From Release Validation Doc:

(Magma 1.7.0 Release Validation.docx)

What?

Who?

ETA?

Notes

Install, Upgrade Validation







AGW BareMetal





does that run the full integ test suite?

AGW Upgrade





does that run the full integ test suite?

Orc8r Fresh Install (Cloudstrapper)







Orc8r Upgrade (Terraform)







Orc8r 1.8 with AGW 1.7







AGW nonsanity integration tests (not executed in CI)







Hardware in the Loop Testing







Sanity

for 4G: Sercomm, Radtonics and Emblasoft

for 5G SA:



Sercomm: Basic Sanity was done on with real radio (Englewood).

Feature

for 4G: Sercomm

for 4G stateless:

for 5G SA:



Sercomm: UE Attach/Detach, data service (speedtest, iperf tcp/udp).

Performance

for 4G: Sercomm, Radtonics and Emblasoft

for 5G SA: Wavelabs



Sercomm: UL/DL performance was verified on with real radio (Englewood 20MHz/2x2 DL MIMO/2CC CA).

Availability

for 4G: Radtonics and Emblasoft

for 5G SA:





Feature Testing







TAU







Docker Access Gateway







IPv6







5G SA

Wavelabs





5G NSA







GTP Echo







Inbound Roaming - Dedicated Bearer







L3 Transport







S8 Unique TEID







Features in Release notes that need to be tested:

Already discussed (see also comments)

4G

see comments

5G

As part of the release plan for 1.8,  Wavelabs is going to cover the 5G SA scenario. This will cover all features delivered as part of 1.7, new features going to be delivered as part of 1.8, throughput and performance cases for 5G SA. Following are the list of test scenarios :

  1. Feature specific testing on daily builds of 1.8, using the commercial testing framework Abbott from Rebecca (This includes the testcases covered by TerraVM in 1.7).   

  2. Spirent related automated testcases. 

  3. Spirent related throughput and performance cases

  4. Verification on one of Real GNB/UE

The Abott tools are already integrated as part of Master branch in CI. It now has to include release 1.8 branch as well. 

Plan is to have the release report of 1.8 on similar lines as 1.7 (with additional new cases).

To be discussed

testing of the python scripts (cf. https://github.com/magma/magma/issues/13606)