Testing

General Approach

Testing Pyramid

The testing pyramid is a proven model that helps in planning and implementing software testing. It consists of three main layers:

1. Unit Tests:
   • The foundation of the pyramid.
   • Test individual functions or methods in isolation.
   • Fast, cheap, and easy to maintain.
2. Widget Tests:
   • The middle layer.
   • Test the interaction between widgets in Flutter.
   • Allow verifying the UI and logic at the widget level.
3. Integration Tests:
   • The top of the pyramid.
   • Test the whole application or large parts of it.
   • Simulate real user interactions and test the application's behavior as a whole.

Test Coverage

Test coverage measures the extent to which your codebase is tested by your test suite. It helps identify which parts of the code are exercised by tests and which parts are not, providing insights into potential gaps in testing and helping ensure that your application is thoroughly validated against various scenarios. High test coverage indicates a lower risk of undetected bugs and higher confidence in code quality.

Automation

Automating repetitive tasks, enforcing code quality, and streamlining development processes are key practices in modern development. By using Git Hooks and CI/CD Pipelines, you can ensure code quality and automate tasks like testing, analysis, etc.

Git Hooks

Git Hooks are scripts that run automatically at specific points in the Git workflow, such as before a commit or after a push. They can be used to enforce certain checks, such as linting or running tests before committing code.

CI/CD Pipelines

Continuous Integration (CI) and Continuous Deployment (CD) pipelines help automate the process of testing, building, and deploying Flutter apps. Using pipelines allows you to enforce quality checks and streamline development workflows.

Getting Started

Types of Testable Code in the Flutter Framework

1. Unit Tests:
   • Test individual functions or methods without dependencies on other parts of the application.
   • Example: Verify that a method adding numbers returns the correct result.

   void main() {
     test('adds two to a given number', () {
       var result = add(2, 3);
       expect(result, 5);
     });
   }
   

2. Widget Tests:
   • Test the interaction between widgets in Flutter.
   • Example: Verify that a Text widget displays the correct text.

   void main() {
     testWidgets('displays a text', (WidgetTester tester) async {
       await tester.pumpWidget(MyApp());
       expect(find.text('Hello, World!'), findsOneWidget);
     });
   }
   

3. Integration Tests:
   • Test the whole application or large parts of it.
   • Example: Verify that the application navigates to the correct screen when a button is pressed.

   void main() {
     testWidgets('navigates to the next screen', (WidgetTester tester) async {
       await tester.pumpWidget(MyApp());
       await tester.tap(find.byType(ElevatedButton));
       await tester.pumpAndSettle();
       expect(find.text('Next Screen'), findsOneWidget);
     });
   }
   

Running Tests

To verify that all tests pass, run the following command in the terminal:

flutter test

Test Coverage

To maintain high code quality and ensure the stability of the app, strive to achieve 100% test coverage across both unit tests and widget tests.

Running Tests and Generating Coverage Report

1. To run your tests and generate a coverage report in Flutter, use the following command:

flutter test --coverage

1. After running this command, you can view the coverage report in coverage/lcov.info file. Tools like the VSCode extensions will help you visualize this coverage data directly in your editor.

Tools for Coverage Report

Several tools can assist you in reaching 100% test coverage:

1. VSCode Flutter Coverage extension

• This extension allows you to visualize test coverage directly in VSCode. It highlights the lines of code that are covered by tests and the ones that are not.
• Install the extension from the VSCode Marketplace.

2. VSCode Coverage Gutters extension

• This extension displays colored markers in the gutter of the VSCode editor, indicating which parts of the code are covered by tests and which are not.
• Install the extension from the VSCode Marketplace.

3. Eli CLI

Use the following command to obtain the overall test coverage percentage:

eli project coverage

Our Approch with Eli Generated App

When you generate a Flutter app using the Eli CLI, everything is already configured for testing, making it easy to begin writing and running tests immediately. Since it's a standard Flutter project, you can run all the usual Flutter test commands to verify and track the performance of your code.

For basic test execution, you can use the same command as you would in any Flutter project:

flutter test

Project Managed by Melos

In addition to the standard Flutter setup, the generated app is also managed by Melos. The melos.yaml configuration file already defines several useful scripts for testing and analysis.

1. ci:analyze: This script runs Flutter’s static analysis tools.

melos run ci:analyze

2. ci:test: This script executes the tests defined in the project, ensuring that all unit and widget tests pass.

melos run ci:test

3. ci:fixcoverage: This script generates a test coverage report and applies fixes related to test coverage.

melos run ci:fixcoverage

These scripts are especially useful for integrating with CI/CD pipelines, as they automate important quality checks like code analysis, running tests, and ensuring sufficient test coverage.

Git Hooks

The project includes pre-configured Git hooks located in the .git/hooks/ folder.

• pre-commit: it runs analyzer
• pre-push: it runs tests

Azure Pipelines

When you generate a new app and set up an Azure project, multiple Azure pipeline files are automatically created in the .azuredevops/ folder.

When it comes to testing, the pipeline of interest is azure-pipelines-pull-request.yml, which runs a Code Quality job on each pull request.

Testing Tools in Flutter

Appium

• Appium is an open-source test automation framework for mobile applications.
• It supports iOS, Android, and Windows applications.
• Appium uses the WebDriver protocol to interact with mobile applications.

Firebase Test Lab

• Firebase Test Lab is a cloud-based app-testing infrastructure provided by Google.
• It allows you to test your app on real devices and configurations in the cloud.
• Firebase Test Lab provides automated testing and detailed test results.

Patrol

• Patrol is a package that builds on top of flutter_test and integration_test to make it easy to control the native UI from Dart test code.