From bb99fab148842db3b4801143983160185c483ecf Mon Sep 17 00:00:00 2001 From: Ilya Shakhat Date: Tue, 14 Jun 2016 14:49:49 +0300 Subject: [PATCH] New scenario for QoS testing All other tenant networking scenarios are extended with new parameters: jitter and packet loss. Change-Id: I3ee35671f4474187daf7ff345e81f9e22fb05024 --- .../test_plans/tenant_networking/plan.rst | 145 ++++++++++++------ .../test_plans/tenant_networking/shaker.rst | 22 ++- 2 files changed, 113 insertions(+), 54 deletions(-) diff --git a/doc/source/test_plans/tenant_networking/plan.rst b/doc/source/test_plans/tenant_networking/plan.rst index 0d9596e..e7e9c21 100644 --- a/doc/source/test_plans/tenant_networking/plan.rst +++ b/doc/source/test_plans/tenant_networking/plan.rst @@ -120,14 +120,16 @@ done during 60 seconds, average values are calculated. List of performance metrics ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== .. _openstack_tenant_networking_test_plan_l2_concurrent: @@ -152,14 +154,16 @@ Test case result is series of following measurements done at different levels of concurrency. The output may be shown in table form and/or as chart showing dependency of parameters from concurrency. -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== .. _openstack_tenant_networking_test_plan_l3_east_west: @@ -178,14 +182,16 @@ average values are calculated. List of performance metrics ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== .. _openstack_tenant_networking_test_plan_l3_east_west_concurrent: @@ -212,14 +218,16 @@ Test case result is series of following measurements done at different levels of concurrency. The output may be shown in table form and/or as chart showing dependency of parameters from concurrency. -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== .. _openstack_tenant_networking_test_plan_l3_north_south: @@ -239,14 +247,16 @@ The measurement is done during 60 seconds, average values are calculated. List of performance metrics ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== .. _openstack_tenant_networking_test_plan_l3_north_south_concurrent: @@ -274,14 +284,53 @@ Test case result is series of following measurements done at different levels of concurrency. The output may be shown in table form and/or as chart showing dependency of parameters from concurrency. -======== =============== ================= ====================================== -Priority Value Measurement Units Description -======== =============== ================= ====================================== -1 Latency ms The network latency -1 TCP bandwidth Mbits/s TCP network bandwidth -2 UDP bandwidth packets per sec Number of UDP packets of 32 bytes size -2 TCP retransmits packets per sec Number of retransmitted TCP packets -======== =============== ================= ====================================== +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== + + +.. _openstack_tenant_networking_test_plan_qos: + +Test Case 7: Neutron QoS testing +-------------------------------- + +Description +^^^^^^^^^^^ + +This test case is used to verify Neutron QoS feature as scale. The feature +allows to limit the traffic bandwidth in a particular network. To simplify +testing instances are deployed in the same network (L2 domain). The target +bandwidth is specified in kBits/s. + +.. note:: + + In order to work Neutron QoS extension must be installed and properly + configured. + + +List of performance metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +======== ================ ================= =========================================== +Priority Value Measurement Units Description +======== ================ ================= =========================================== +1 Latency ms The network latency +1 TCP bandwidth Mbits/s TCP network bandwidth +1 UDP bandwidth packets per sec Number of UDP packets with 32 bytes payload +2 UDP delay jitter ms Packet delay variation +2 UDP packet loss % Percentage of lost UDP packets +2 TCP retransmits packets per sec Number of retransmitted TCP packets +======== ================ ================= =========================================== + +It's expected that achieved TCP bandwidth is not higher that the one set in +Neutron QoS policy. Tools diff --git a/doc/source/test_plans/tenant_networking/shaker.rst b/doc/source/test_plans/tenant_networking/shaker.rst index 197e0c7..df56cf6 100644 --- a/doc/source/test_plans/tenant_networking/shaker.rst +++ b/doc/source/test_plans/tenant_networking/shaker.rst @@ -53,7 +53,7 @@ Test Case 1: L2 instance-to-instance performance **Execution**:: - shaker --server-endpoint --scenario networking/perf_l2 --book + shaker --server-endpoint --scenario openstack/perf_l2 --book Test Case 2: L2 concurrent performance @@ -64,7 +64,7 @@ Test Case 2: L2 concurrent performance **Execution**:: - shaker --server-endpoint --scenario networking/full_l2 --book + shaker --server-endpoint --scenario openstack/full_l2 --book Test Case 3: L3 east-west instance-to-instance performance @@ -74,7 +74,7 @@ Test Case 3: L3 east-west instance-to-instance performance **Execution**:: - shaker --server-endpoint --scenario networking/perf_l3_east_west --book + shaker --server-endpoint --scenario openstack/perf_l3_east_west --book Test Case 4: L3 east-west concurrent performance @@ -84,7 +84,7 @@ Test Case 4: L3 east-west concurrent performance **Execution**:: - shaker --server-endpoint --scenario networking/full_l3_east_west --book + shaker --server-endpoint --scenario openstack/full_l3_east_west --book Test Case 5: L3 north-south instance-to-instance performance @@ -94,7 +94,7 @@ Test Case 5: L3 north-south instance-to-instance performance **Execution**:: - shaker --server-endpoint --scenario networking/perf_l3_north_south --book + shaker --server-endpoint --scenario openstack/perf_l3_north_south --book Test Case 6: L3 north_south concurrent performance @@ -104,7 +104,17 @@ Test Case 6: L3 north_south concurrent performance **Execution**:: - shaker --server-endpoint --scenario networking/full_l3_north_south --book + shaker --server-endpoint --scenario openstack/full_l3_north_south --book + + +Test Case 7: Neutron QoS testing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**Test case specification**: :ref:`openstack_tenant_networking_test_plan_qos` + +**Execution**:: + + shaker --server-endpoint --scenario openstack/qos/perf_l2 --book .. references: