This module provides a way to enhance your existing Artillery Performance tests with trace-based testing. It allows you to create, run, and orchestrate Tracetest Tests based on your existing Artillery Test scripts with minimal configuration.
To find a detailed step-by-step tutorial on how to use this module head out to the main integration page in our docs.
config:
target: my_target
tracetest:
token: <YOUR_TRACETEST_API_TOKEN>
phases:
- duration: 1
arrivalRate: 1
engines:
tracetest: {}
scenarios:
- name: tracetest_engine_test
engine: tracetest
flow:
- test:
definition: import-pokemon.yaml
runInfo:
variables:
- key: ENDPOINT
value: http://api:8081
- key: POKEMON_ID
value: "6"
- summary:
format: "pretty"> quick-start-artillery@1.0.0 test:engine
> artillery run engine-test.yaml
Test run id: t47ez_f4gmj899tr4nnp465h38d4mnker7z_w8yp
Phase started: unnamed (index: 0, duration: 2s) 09:29:33(-0600)
Phase completed: unnamed (index: 0, duration: 2s) 09:29:35(-0600)
--------------------------------------
Metrics for period to: 09:29:40(-0600) (width: 1.203s)
--------------------------------------
vusers.created: ................................................................ 10
vusers.created_by_name.tracetest_engine_test: .................................. 10
Warning: multiple batches of metrics for period 1709825370000 2024-03-07T15:29:30.000Z
⠏ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/70) - trace id: 4a2e41bf5b497a643056c8a08d122d82
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/74) - trace id: a00ff614a06be196ec5cf74ace906b3e
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠸ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/76) - trace id: 1fb3ded60f582ecfd151a019df2479cd
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠹ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/78) - trace id: 750df8c1d55ed6a4e18f4b2b8b573a14
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠦ ✔ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/71) - trace id: c3bdcfb86ca77a9455b29882f593848a
⠏ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/79) - trace id: 76ecf991c6eb37c2a0dee857a699dc76
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/72) - trace id: cece9c79434b3313bf23150bc967e82f
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠇ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/77) - trace id: b3f3682eb328195ede902177ff4170a1
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠸ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/75) - trace id: c2d4624f20060565b0b5e9f97ad7e7a7
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
⠙ ✘ Artillery Engine: Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/artillery-engine-import-pokemon/run/73) - trace id: 878bb73b5234f764f4d289bf657b40cf
> All HTTP Spans: Status code is 200 (span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"])
• Expected: attr:http.status_code = 200
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.status_code" from. To fix this error, create a selector matching at least one span.
> span[tracetest.span.type="http" name="GET" http.method="GET"]
• Expected: attr:http.route = "/api/v2/pokemon/${var:POKEMON_ID}"
No Spans
Received: resolution error: there are no matching spans to retrieve the attribute "http.route" from. To fix this error, create a selector matching at least one span.
> All Database Spans: Processing time is less than 1s (span[tracetest.span.type="database"])
--------------------------------------
Metrics for period to: 09:30:00(-0600) (width: 0.015s)
--------------------------------------
tracetest.tests_failed: ........................................................ 1
vusers.completed: .............................................................. 1
vusers.failed: ................................................................. 0
vusers.session_length:
min: ......................................................................... 21797.1
max: ......................................................................... 21797.1
mean: ........................................................................ 21797.1
median: ...................................................................... 21813.5
p95: ......................................................................... 21813.5
p99: ......................................................................... 21813.5
All VUs finished. Total time: 34 seconds
--------------------------------
Summary report @ 09:30:09(-0600)
--------------------------------
tracetest.tests_failed: ........................................................ 9
tracetest.tests_succeeded: ..................................................... 1
vusers.completed: .............................................................. 10
vusers.created: ................................................................ 10
vusers.created_by_name.tracetest_engine_test: .................................. 10
vusers.failed: ................................................................. 0
vusers.session_length:
min: ......................................................................... 21797.1
max: ......................................................................... 33447
mean: ........................................................................ 28651.1
median: ...................................................................... 28862.3
p95: ......................................................................... 32542.3
p99: ......................................................................... 32542.3- For a detailed step-by-step tutorial on how to use this module: visit https://docs.tracetest.io/tools-and-integrations/artillery-engine
- If you want to get in touch with the Tracetest team: visit https://dub.sh/tracetest-community