x/xenapi-os-testing
Code Issues Proposed changes

Remove some tests from the exclusion list.

Browse Source 
We can remove some tests now:
1. some tests are obsolete:
tempest.api.volume.test_volumes_actions.*XML.test_volume_upload.*'
tempest.thirdparty.boto.*
2. fixed by recent change:
tempest.scenario.test_shelve_instance.TestShelveInstance.*
tempest.scenario.test_shelve_instance.TestShelveInstance.*
tempest.scenario.test_minimum_basic.TestMinimumBasicScenaro.*
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.*
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.*
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.*
And this patch also make each item as one shell cmdline so that it
is easier to add comments to explain why each item is there.

Change-Id: I0e3a6ec29fe7e803545d98d7ae1a0b351370c42f
This commit is contained in:
Jianghua Wang 2016-08-30 14:36:23 +08:00
parent f70b0b535c
commit 47063c0fb4
1 changed files with 53 additions and 62 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

115
tempest_exclusion_list
View File

@ -1,64 +1,55 @@
COMMON_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\]'\
'|.*tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload.*'\
'|.*tempest.api.volume.test_volumes_actions.VolumesActionsTestXML.test_volume_upload.*'\
'|.*tempest.api.volume.test_volumes_actions.VolumesV1ActionsTest.test_volume_upload.*'\
'|.*tempest.api.volume.test_volumes_actions.VolumesV1ActionsTestXML.test_volume_upload.*'\
'|.*tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_volume_upload.*'\
'|.*tempest.api.volume.test_volumes_actions.VolumesV2ActionsTestXML.test_volume_upload.*'\
'|.*tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_cryptsetup.*'\
'|.*tempest.scenario.test_encrypted_cinder_volumes.TestEncryptedCinderVolumes.test_encrypted_cinder_volumes_luks.*'\
'|.*tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario.*'\
'|.*tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_instance.*'\
'|.*tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance.*'\
'|.*tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern.*'\
'|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_create_ebs_image_and_check_boot.*'\
'|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern.*'\
'|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_create_ebs_image_and_check_boot.*'\
'|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern.*'\
'|.*tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes.*'\
'|.*tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_idempotent_instances.*'\
'|.*tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance.*'\
'|.*tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance_with_tags.*'\
'|.*tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_terminate_instance.*'\
'|.*tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest.test_create_get_delete.*'\
'|.*tempest.thirdparty.boto.test_ec2_volumes.EC2VolumesTest.test_create_volume_from_snapshot.*'\
'|.*tempest.thirdparty.boto.test_s3_ec2_images.S3ImagesTest.test_register_get_deregister_aki_image.*'\
'|.*tempest.thirdparty.boto.test_s3_ec2_images.S3ImagesTest.test_register_get_deregister_ami_image.*'\
'|.*tempest.thirdparty.boto.test_s3_ec2_images.S3ImagesTest.test_register_get_deregister_ari_image.*'
COMMON_TEMPEST_REGEX="(?!.*\[.*\bslow\b.*\]"
NOVA_NETWORK_TEMPEST_REGEX=$COMMON_TEMPEST_REGEX\
')(^tempest\.(api|scenario|thirdparty))'
# waiting for the fix on https://bugs.launchpad.net/nova/+bug/1616938
COMMON_TEMPEST_REGEX="$COMMON_TEMPEST_REGEX|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_create_ebs_image_and_check_boot.*"
COMMON_TEMPEST_REGEX="$COMMON_TEMPEST_REGEX|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_create_ebs_image_and_check_boot.*"
NEUTRON_NETWORK_TEMPEST_REGEX=$COMMON_TEMPEST_REGEX\
'|.*tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces.*'\
'|.*tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers.*'\
'|.*tempest.api.compute.servers.test_server_personality.ServerPersonalityTestJSON.test_rebuild_server_with_personality.*'\
'|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server_in_stop_state.*'\
'|.*tempest.api.compute.admin.test_auto_allocate_network.AutoAllocateNetworkTest.test_server_multi_create_auto_allocate.*'\
'|.*tempest.api.compute.admin.test_servers.ServersAdminTestJSON.test_rebuild_server_in_error_state.*'\
'|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server.*'\
'|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server_with_volume_attached.*'\
'|.*tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_rebuild_server_with_auto_disk_config.*'\
'|.*tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_rebuild_server_with_manual_disk_config.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_resize.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic.*'\
'|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops.*'\
'|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port.*'\
'|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_in_tenant_traffic.*'\
'|.*tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling.*'\
'|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state.*'\
'|.*tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac.*'\
'|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group.*'\
'|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state.*'\
'|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group.*'\
')(^tempest\.(api|scenario|thirdparty))'
NOVA_NETWORK_TEMPEST_REGEX=$COMMON_TEMPEST_REGEX
# Wait untill resolve the disk IO bottleneck with nova CI cloud
NOVA_NETWORK_TEMPEST_REGEX="$NOVA_NETWORK_TEMPEST_REGEX|.*tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload.*"
NOVA_NETWORK_TEMPEST_REGEX="$NOVA_NETWORK_TEMPEST_REGEX|.*tempest.api.volume.test_volumes_actions.VolumesV1ActionsTest.test_volume_upload.*"
NOVA_NETWORK_TEMPEST_REGEX="$NOVA_NETWORK_TEMPEST_REGEX|.*tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_volume_upload.*"
NOVA_NETWORK_TEMPEST_REGEX="$NOVA_NETWORK_TEMPEST_REGEX)(^tempest\.(api|scenario|thirdparty))"
NEUTRON_NETWORK_TEMPEST_REGEX=$COMMON_TEMPEST_REGEX
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_server_personality.ServerPersonalityTestJSON.test_rebuild_server_with_personality.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server_in_stop_state.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.admin.test_auto_allocate_network.AutoAllocateNetworkTest.test_server_multi_create_auto_allocate.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.admin.test_servers.ServersAdminTestJSON.test_rebuild_server_in_error_state.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server_with_volume_attached.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_rebuild_server_with_auto_disk_config.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_rebuild_server_with_manual_disk_config.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_connectivity_between_vms_on_different_networks.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_resize.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_in_tenant_traffic.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group.*"
# Need resolve the timeout issue when access from floating IP.
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_instance.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern.*"
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX)(^tempest\.(api|scenario|thirdparty))"