重温UAT

UAT我们在项目还是经常听到，我们今天重温下：

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.

https://en.wikipedia.org/wiki/Acceptance_testing#User_acceptance_testing

Acceptance Tests An essential property of a software requirement is that it should be possible to validate that the finished product satisfies it. Requirements that cannot be validated are really just “wishes.” An important task is therefore planning how to verify each requirement. In most cases, designing acceptance tests does this for how end-users typically conduct business using the system. Identifying and designing acceptance tests may be difficult for nonfunctional requirements. To be validated, they must first be analyzed and decomposed to the point where they can be expressed quantitatively.

---SWEBOK V3

Formal testing with respect to user needs, requirements, and business processes conducted to determine whether a system satisfies the acceptance criteria [2] and to enable the user, customers or other authorized entity to determine whether to accept the system.

— Standard Glossary of Terms used in Software Testing

Key points to be considered while designing Acceptance Test Plan:

It should be Detailed and Specific. Must include only what is required for testing and what information is necessary for the team to carry out testing.
It should be Clear and Concise. No ambiguity. If at all there is something which may lead to confusion, then elaborate it, but keep it short and effective.
Each and every component in the document should be written by keeping only the Business Requirements in mind.
Reliable and adaptable – It should be updatable as required in the future releases.
Consistent – It should not have more changes in the future.
Follow the template provided by the Organization or Customer.

Key points to be remembered before writing acceptance tests:

Keep all the reference documents in place: Software Requirement Specification, Business Requirements Document, Use Cases, User Stories, Data matrix (in case of logic involved) etc.
Focus only on business requirements (testable business requirements).
Clear all the doubts, queries on the business requirements at the earliest.
Make sure there are no changes on the requirements for the current release at least.

Case : Purchasing Product

Purchasing of the product usually has the general flow.

Some general scenarios what end-users look at are listed here:

Pre-condition: User should be logged in to the application.

Test 1: Product Details, verify if a User is able to:

View the Product details page.
View all the sub-sections in the Product details page (Description, Feature, Brand information, etc.).
Select the Quantity of the product, Color, Size, etc. as available in the Product details page.
Navigate to the category, sub-category pages from the Product Details page (if available in Product details page).
Navigate to the other Product’s details page (if provided relevant products section).
View comments and ratings on the product.
Sort Comments of the Product based on ratings.
View overall rating of the Product.
Add Comment on the Product.
Update his/her comment on the product.
Delete his/her comment on the product (if provided).

Test 2: Add to Cart, verify if a User is:

Able to add the product to Cart:

Through Product details page.
Through Product list page.

Able to add required quantity to the cart ( 1 to max limit set).
Not able to add the product to the Cart if Out-of-Stock.

Test 3: In the Cart Page, verify if a User is able to:

View the Product in the Cart with Price details for added quantity.
Update quantity (1 to max limit set).
Remove the Product from Cart.
Navigate back to shopping.
Continue to Checkout.
View Empty Cart when no product is added,

Test 4: In the Account Details Page, verify if a User is able to:

Continue with the existing Shipping details.
Update Shipping address.
Add new shipping address.
Continue with the existing Phone number.
Update Phone number for the order.
Add new Phone number for the order.
Navigate back to the Cart page.
Navigate to the Payment page.

Test 5: On the Payments Page, verify if a User is able to:

Verify the correctness of the amount to be billed.
Process the order with all the available options (One option for each separate order).
Process transaction Successfully. Go to the Order Confirmation page.
Transaction Failure (Even though this is negative testing, it should be considered as a major scenario).
Apply coupons:

Valid Coupons – Success. Here verify the change in the amount to be billed.
Invalid coupons – Failure

Expired coupons – Failure.
Navigate back to the Account details page.

Reviewing Acceptance Tests

Reviewing Acceptance tests is an important task as it needs to be correct and to-the-point with respect to the business requirements. As these may be conducted by Customers themselves and/or end-users, it is very much necessary to be complete, non-ambiguous, correct, and detailed enough for anyone to understand and execute.

Reviewing Acceptance tests has to be done by Business Analysts, Customers and any review comments should be incorporated into high priority.

At the individual test level, the review should be done against the below:

Whether test covers the business requirement or not.
Are the pre-conditions clear?
Are the test steps easy to understand and detailed?
Is the expected result correct and clear?
Is it mapped to the Business requirements for traceability?
Is the test complete enough to cover the particular flow or usage?
Is the particular test required as part of acceptance testing.
Is there some verification point that is not needed for acceptance testing.
Is it purely functional or any GUI is covered within (It should be only Functional).
Is the special input data necessary? If yes, is it provided for particulars?

On the whole, entire Acceptance tests suite review should cover:

Bi-directional traceability: Business requirements to Tests AND Tests to Business requirements.
Is each and every business requirement covered?
Is every business requirement covered by one or more tests?
Are the business rules covered?
Is the special data case handled?
How many tests are written to cover each requirement or rule?
Can the tests be grouped together and classified for flows.
Are the tests sequenced properly so that the execution is efficient?

UAT In Agile Environment

The agile environment is more dynamic in nature. In an agile world, business users will be involved throughout the project sprints and the project would be enhanced based on the feedback loops from them.

At the beginning of the project, business users would be the key stakeholders to provide requirement thereby updating the product backlog. During the end of each sprint, business users would participate in the sprint demo and would be available for providing any feedback.

Moreover, a UAT phase would be planned before the sprint completion where the business users would do their validations.

The feedbacks which are received during sprint demo and sprint UAT, are collated and added back to the product backlog which is constantly reviewed and prioritized. Thus in an agile world, the business users are more close to the project and they evaluate the same for its use more frequently unlike the traditional waterfall projects.

Acceptance Testing and V-Model

In VModel, User acceptance testing corresponds to the requirement phase of the Software Development life cycle(SDLC).

Prerequisites of User Acceptance Testing:

Following are the entry criteria for User Acceptance Testing:

Business Requirements must be available.
Application Code should be fully developed
Unit Testing, Integration Testing & System Testing should be completed
No Showstoppers, High, Medium defects in System Integration Test Phase -
Only Cosmetic error is acceptable before UAT
Regression Testing should be completed with no major defects
All the reported defects should be fixed and tested before UAT
Traceability matrix for all testing should be completed
UAT Environment must be ready
Sign off mail or communication from System Testing Team that the system is ready for UAT execution

How to do UAT Testing

UAT is done by the intended users of the system or software. This type of Software Testing usually happens at the client location which is known as Beta Testing. Once Entry criteria for UAT are satisfied, following are the tasks need to be performed by the testers:

UAT Process

Analysis of Business Requirements
Creation of UAT test plan
Identify Test Scenarios
Create UAT Test Cases
Preparation of Test Data(Production like Data)
Run the Test cases
Record the Results
Confirm business objectives

Step 1) Analysis of Business Requirements

One of the most important activities in the UAT is to identify and develop test scenarios. These test scenarios are derived from the following documents:

Project Charter
Business Use Cases
Process Flow Diagrams
Business Requirements Document(BRD)
System Requirements Specification(SRS)

Step 2) Creation of UAT Plan:

The UAT test plan outlines the strategy that will be used to verify and ensure an application meets its business requirements. It documents entry and exit criteria for UAT, Test scenarios and test cases approach and timelines of testing.

Step 3) Identify Test Scenarios and Test Cases:

Identify the test scenarios with respect to high-level business process and create test cases with clear test steps. Test Cases should sufficiently cover most of the UAT scenarios. Business Use cases are input for creating the test cases.

Step 4) Preparation of Test Data:

It is best advised to use live data for UAT. Data should be scrambled for privacy and security reasons. Tester should be familiar with the database flow.

Step 5) Run and record the results:

Execute test cases and report bugs if any. Re-test bugs once fixed. Test Management tools can be used for execution.

Step 6) Confirm Business Objectives met:

Business Analysts or UAT Testers needs to send a sign off mail after the UAT testing. After sign-off, the product is good to go for production. Deliverables for UAT testing are Test Plan, UAT Scenarios and Test Cases, Test Results and Defect Log

Exit criteria for UAT:

Before moving into production, following needs to be considered:

No critical defects open
Business process works satisfactorily
UAT Sign off meeting with all stakeholders

Best Practices:

Following points needs to be considered to make UAT Success:

Prepare UAT plan early in the project life cycle
Prepare Checklist before the UAT starts
Conduct Pre-UAT session during System Testing phase itself
Set the expectation and define the scope of UAT clearly
Test End to End business flow and avoid system tests
Test the system or application with real-world scenarios and data
Think as an Unknown user to the system
Perform Usability Testing
Conduct Feedback session and meeting before moving to production

Acceptance Testing Process

In V-Model, AT phase is in parallel to the Requirements phase.

Actual AT process goes as shown below:

Business Requirements Analysis

Business requirements are analyzed by referring all the available documents within the project.

Some of which are:

System Requirement Specifications
Business Requirements Document
Use Cases
Workflow diagrams
Designed data matrix

Design Acceptance Test Plan

There are certain items to be documented in the Acceptance Test Plan.

Let’s take a look at some of them:

Acceptance Testing strategy and approach.
Entry and exit criteria should be well-defined.
The scope of AT should be well-mentioned and it has to cover only the business requirements.
Acceptance test design approach should be detailed so as anyone writing tests can easily understand the way in which it has to be written.
Test Bed set up, actual testing schedule/timelines should be mentioned.
As testing is conducted by different stakeholders, details on logging bug should be mentioned as the stakeholders may not be aware of the procedure followed.

Design and Review Acceptance Tests

Acceptance tests should be written at a scenario level mentioning what has to be done (not in-detail to include how to do). These should be written only for the identified areas of scope for business requirements, and each and every test has to be mapped to its referencing requirement.

All the written acceptance tests have to be reviewed to achieve high coverage on business requirements.

This is to make sure that any other tests apart from scope mentioned are not involved so that testing lies within the scheduled timelines.

Acceptance Test Bed Set up

Test Bed should be set up similar to a Production environment. Very high-level checks are required to confirm on environment stability and usage. Share the credentials to use the environment only with a stakeholder who is performing this testing.

Acceptance Test Data Set Up

Production data has to be prepared/populated as test data in the systems. Also, there should be a detailed document in such a way that the data has to be used for testing.

Do not have the test data like TestName1, TestCity1, etc., Instead have Albert, Mexico, etc. This gives a rich experience of real-time data and testing will be up-to-the-point.

Acceptance Test Execution

Designed Acceptance tests have to be executed on the environment at this step. Ideally, all the tests should pass at the first attempt itself. There should be no functional bugs arising out of Acceptance testing, if any, then they should be reported at a high priority to be fixed.

Again, bugs fixed have to be verified and closed as a high priority task. Test execution report has to be shared on a daily basis.

Bugs logged in this phase should be discussed in a bug-triage meeting and has to undergo Root Cause Analysis procedure. This is the only point where acceptance testing assess whether all the business requirements are actually met by the product or not.

Business Decision

There comes out a Go/No-Go decision for the product to be launched in Production. Go decision will take the product ahead to be released to the market. No-Go decision marks the product as Failure.

Few factors of No-Go Decision:

Poor Quality of the product.
Too Many open Functional Bugs.
Deviation from business requirements.
Not up to the market standards and needs enhancements to match the current market standards.

Success factors for This Testing

Once this test is planned, prepare a checklist which increases the success rate of it. There are some action items that are to be followed before Acceptance test starts.

They are:

Have a well-defined scope and make sure there is a business need for the scope identified for this testing.
Execute Acceptance tests in System testing phase itself at least once.
Perform extensive ad-hoc testing for each of the acceptance test scenario

今天先到这儿，希望对云原生，技术领导力，企业管理，系统架构设计与评估，团队管理, 项目管理, 产品管管，团队建设有参考作用 , 您可能感兴趣的文章:
领导人怎样带领好团队
构建创业公司突击小团队
国际化环境下系统架构演化
微服务架构设计
视频直播平台的系统架构演化
微服务与Docker介绍
Docker与CI持续集成/CD
互联网电商购物车架构演变案例
互联网业务场景下消息队列架构
互联网高效研发团队管理演进之一
消息系统架构设计演进
互联网电商搜索架构演化之一
企业信息化与软件工程的迷思
企业项目化管理介绍
软件项目成功之要素
人际沟通风格介绍一
精益IT组织与分享式领导
学习型组织与企业
企业创新文化与等级观念
组织目标与个人目标
初创公司人才招聘与管理
人才公司环境与企业文化
企业文化、团队文化与知识共享
高效能的团队建设
项目管理沟通计划
构建高效的研发与自动化运维
某大型电商云平台实践
互联网数据库架构设计思路
IT基础架构规划方案一(网络系统规划)
餐饮行业解决方案之客户分析流程
餐饮行业解决方案之采购战略制定与实施流程
餐饮行业解决方案之业务设计流程
供应链需求调研CheckList
企业应用之性能实时度量系统演变

如有想了解更多软件设计与架构, 系统IT,企业信息化, 团队管理资讯，请关注我的微信订阅号：

作者：Petter Liu
出处：http://www.cnblogs.com/wintersun/
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。该文章也同时发布在我的独立博客中-Petter Liu Blog。