In my look at different testing tools, I’ve focused mostly on automation tools, but today I want to focus on a tool that is exclusively focused on performance testing. Yes, I wrote about JMeter previously which is also a very good performance testing tool, but it can also serve as a good functional test automation tool on its own. Today though I’m going to be looking at k6.
I’ve written about k6 before, largely because I’m a big fan because it easily integrates into the CI/CD pipelines and allows for a form of load and performance testing to be conducted early on in the development cycle. Whereas many other performance tools require a fair amount of processing and therefore will often require a dedicated environment to sufficiently execute performance tests, k6 can spin up multiple instances of a script with relatively low processing power, (partly thanks to it being developed in Go) allowing it to integrate into the pipeline process more seamlessly without adversely affecting pipeline performance.
K6 is able to achieve this through its simplicity It doesn’t offer a lot of features compared to other tools and its scripting framework is not really powerful, but in keeping things light, it can scale more effectively and deliver the results early.
Below are some details of how k6 works:
Command Execution: When you run a k6 test, it starts by executing the specified test script using the k6 run command. k6 uses the V8 JavaScript engine (the same engine that powers Google Chrome) to execute the JavaScript code in your test script.
Virtual Users (VUs): The core concept in k6 is virtual users (VUs). Each VU is a separate JavaScript runtime that executes your test script independently. This is how k6 simulates multiple users simultaneously. The number of VUs is defined in your test script or via command-line arguments.
Event Loop: Within each VU, k6 uses an event loop to manage the execution of the test script. The event loop processes tasks asynchronously, such as making HTTP requests, handling timers, and managing WebSocket connections.
HTTP Requests: When your test script issues an HTTP request using k6's HTTP API, k6 uses a Go-based HTTP client under the hood to send the request. This client manages the connection pool, request/response handling, and supports a range of HTTP methods, headers, and cookies.
Response Handling: Once k6 receives a response from the server, it parses the response data, including headers, body, and status codes. You can access this data in your test script for custom handling or to extract information.
Metrics Collection: k6 collects various performance metrics during the test execution. These metrics include response times, throughput, error rates, and custom metrics defined in your script. k6 exposes an API to access and record these metrics during the test.
Output Formats: k6 supports multiple output formats (e.g., plain text, JSON, and InfluxDB) for storing the collected metrics and results. You can specify the desired output format using command-line options.
Parallel Execution: k6 can execute multiple VUs concurrently, which allows you to simulate a large number of users. The actual level of concurrency depends on your hardware capabilities and the configuration settings.
Distributed Testing: For distributed testing, you can run k6 on multiple machines, each running its set of VUs. In distributed mode, k6 uses the k6 Cloud service or the k6 Load Impact service to coordinate the test execution across the different instances. This allows you to test applications under even heavier loads.
Environment Variables and Configuration: k6 allows you to configure test parameters and settings using environment variables and configuration files. These can be used to parameterize your test script and control various aspects of test execution.
Real-Time Reporting: k6 provides real-time reporting of test results to the console, enabling you to monitor the test's progress and identify any immediate issues or anomalies.
Custom Libraries and Modules: You can extend k6's functionality by using custom libraries and modules. These can be written in JavaScript or Go and can be used for tasks such as generating test data or handling complex logic.
And to showcase just how easy scripting is in k6, I’ve provided a very basic example:
import http from 'k6/http';
import { sleep, check } from 'k6';
export let options = {
vus: 5, // Number of virtual users (VUs)
duration: '30s', // Test duration
stages: [
{ duration: '10s', target: 10 }, // 10 VUs for 10 seconds
{ duration: '20s', target: 50 }, // 50 VUs for 20 seconds
{ duration: '10s', target: 0 }, // Ramp down to 0 VUs ],
};
export default function () {
const res = http.get('https://example.com');
check(res, {
'status is 200': (r) => r.status === 200,
});
sleep(1);
}I’ll also explain what is happening in more detail below:
Importing Modules: The script begins by importing necessary k6 modules. http is used for making HTTP requests, and sleep and check are used for controlling the virtual users and performing response checks, respectively.
options Object: This section configures the test options. It specifies the number of virtual users (vus), the total duration of the test (duration), and the ramp-up stages using the stages array.
vus: 5 indicates that the test will start with 5 virtual users.
duration: '30s' specifies that the test will run for 30 seconds.
The stages array defines the ramp-up and ramp-down stages:
The first stage, { duration: '10s', target: 10 }, means that for the first 10 seconds, the number of virtual users will gradually increase to 10.
The second stage, { duration: '20s', target: 50 }, indicates that for the next 20 seconds, the number of virtual users will increase to 50.
The third stage, { duration: '10s', target: 0 }, ramps down the virtual users to 0 over 10 seconds.
Test Function: The export default function () block is the main test function that each virtual user executes. In this case, it performs the following actions:
Makes an HTTP GET request to 'https://example.com'.
Checks if the response status code is 200 using the check function.
Pauses for 1 second using the sleep function to simulate user think time.
This script simulates a scenario where the number of virtual users increases from 5 to 10 over the first 10 seconds, then from 10 to 50 over the next 20 seconds, and finally ramps down to 0 over the last 10 seconds. During this time, each virtual user is making HTTP requests to 'https://example.com' and checking if the responses have a status code of 200.
The options object and the defined test function work together to simulate a gradual increase and decrease in user load, making it suitable for load testing and assessing how your application handles changing levels of traffic. You can customize the script to fit your specific testing needs by modifying the URL, adding more HTTP requests, and defining additional checks and custom metrics.
As you can see, k6 is a very simple scripting language to use, which is part of what makes it so popular. However, people with experience in performance testing will tell you that scripting is a small component of the process and it’s gathering the underlying g performance of a system and analysing it that is where the majority of issues are then resolved. Admittedly, this is the weakness of k6, as it doesn’t try and gather any of this information, and while it can provide decent reporting on the execution of its vus and scripts, it requires teams to monitor the underlying application or container performance themselves.
Additionally, k6 doesn’t track trends over time, though metrics like response times, throughput, error rates, and various other custom metrics that you define, can be pushed into a database and visualized through tools like Grafana to provide a better view of system performance.
Below is a brief overview of the history of the tool:
Initial Development (2016-2017): The project that eventually became k6 was initially developed by Load Impact, a company specializing in cloud-based load and performance testing. They had been working on improving their internal load testing tool and decided to rewrite it from scratch to make it more developer-friendly.
Launch of Open Source (2017): In 2017, Load Impact released the core of their load testing tool as open-source software under the name "k6" (pronounced "k-six"). It was written in Go and designed with a strong focus on ease of use and developer-centric scripting.
Growth of the Community (2017-2018): k6 quickly gained popularity within the developer and testing communities due to its simplicity and scalability. An active community of users and contributors started to form around the project.
Commercial Offering (2018): While the open-source version of k6 remained free to use, Load Impact also introduced a commercial version known as "k6 Cloud" (formerly Load Impact). The cloud-based offering provided additional features and made it easier to scale tests across multiple geographic regions.
K6 as a Standalone Company (2020): In 2020, k6 became a standalone company, separate from Load Impact. The k6 team continued to focus on improving the tool and expanding its capabilities.
Release of k6 v0.27.0 (2021): In September 2021, k6 released version 0.27.0, which included several significant enhancements, such as custom metrics and distributed testing capabilities.
Ongoing Development and Adoption (2022-2023): k6 continued to evolve, with ongoing development, feature enhancements, and the growth of the user community. The tool's popularity continued to rise, with many organizations and individuals adopting it for performance testing.
Below is a list of pros and cons of k6:
Pros:
Open Source: k6 is an open-source tool, which means it's free to use and has an active community of users and contributors. You can customize and extend it to meet your specific testing needs.
Developer-Friendly: k6 uses JavaScript for scripting, which is a language familiar to many developers. This makes it easy for developers to create and maintain test scripts without needing to learn a new scripting language.
Concurrent Virtual Users (VUs): k6 allows you to simulate a high number of virtual users (VUs) concurrently. Each VU can execute the test script independently, enabling realistic load testing.
Distributed Testing: You can distribute test execution across multiple machines, making it possible to test the scalability of your application effectively.
Realistic Scenarios: k6 allows you to create complex test scenarios to simulate various user behaviors, making it suitable for testing real-world use cases.
WebSocket Support: Support for WebSocket testing is a valuable feature for applications that rely on real-time communication.
Metrics and Analysis: k6 provides extensive performance metrics, including response times, throughput, and custom metrics. This data can be analyzed and visualized for better insights into your application's performance.
CI/CD Integration: Integration with CI/CD pipelines is straightforward, enabling automated performance testing as part of your development workflow.
Customization: You can customize k6 to suit your specific needs by writing custom functions, handling authentication, and generating dynamic data.
Cons:
JavaScript Proficiency: While using JavaScript is an advantage for many developers, those who are not familiar with JavaScript may face a steeper learning curve.
Limited Protocols: k6 primarily focuses on HTTP and WebSocket testing. If your application relies heavily on other protocols, you may need additional tools for comprehensive testing.
Resource Intensive: Running a large number of concurrent VUs or distributed tests may require significant computing resources, which could be a limitation for some organizations.
Learning Curve: Although it's developer-friendly, k6 may still have a learning curve for those who are new to performance testing concepts or load testing in general.
Limited Reporting: While k6 provides basic result output formats, more advanced reporting, and visualization may require integration with third-party tools like Grafana or InfluxDB.
Less Mature Ecosystem: Compared to some other load testing tools, k6's ecosystem of plugins, integrations, and community resources may be less extensive, but it's growing over time.
If you’re looking to shift your performance testing left and get results as early as possible, then k6 is definitely your option, as it can operate at that unit level and provide useful feedback to developers early in the development cycle. It’s probably not as good at providing an overview of bigger system performance and you may want to partner it with another performance tool when looking at end-to-end application performance. For microservices, APIs, and code performance though k6 is highly recommended for early performance feedback.