Engineering Council Test Reliability Report

Scope aligned with Slack channel #dezvoltare, covering 2026-03-14 to 2026-03-21. Metrics and timings are sourced from GitLab pipelines, jobs, and test-report artifacts for the daily 6 PM regression suite and the production smoke suite. Trend charts use daily buckets across this window.

Executive Snapshot

8
Daily Runs
6/8
Daily Green
19m 44s
Avg Daily Runtime
17
Smoke Attempts
12/17
Smoke Green
2m 40s
Avg Smoke Runtime
3m 03s
Median Smoke Time
3
Current Green Streak

Council Narrative

  • Daily suite reliability finished strong: the last 3 daily runs were all green.
  • In the last 10 daily runs, total test volume grew from 1,188 to 1,189.
  • Library has the highest strict failure ratio at 0.00%, while Billing has the broadest non-pass footprint at 0.00% and showed up in 0 failed runs.
  • The average daily-suite runtime, measured from GitLab start and finish timestamps, was 19m 44s.
  • Smoke runs stayed fast when healthy: average duration was 2m 40s and the median passing run was 3m 03s.

Category Failure Ratios

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

Category Failure RatioBilling0.00%Web0.03%Frontend0.00%Library0.00%
Category Non-pass Ratio (fail + pending + skipped)Billing0.00%Web0.03%Frontend0.00%Library0.00%
Daily Daily Suite Status0000103-1403-1603-1803-2003-21
Daily Smoke Attempts0134603-1403-1603-1803-2003-21
Daily Average Daily Suite Runtime19m 13s19m 40s20m 08s20m 36s21m 03s03-1403-1603-1803-2003-21
Daily Average Smoke Runtime0m 00s0m 47s1m 35s2m 22s3m 10s03-1403-1603-1803-2003-21
Daily Suite Total Test Growth (Recent 8 Runs)1188118811881188118903-1403-1603-1803-2003-21

Category Aggregate Table

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

CategoryTotalFailedPendingSkippedFailure RatioNon-pass RatioRuns With Failures
Billing8640000.00%0.00%0
Web59442000.03%0.03%2
Frontend20110000.00%0.00%0
Library6880000.00%0.00%0
CatFailF%NP%Tot
Billing
Pend 0Skip 0Runs 0
0
0.00%
0.00%
864
Web
Pend 0Skip 0Runs 2
2
0.03%
0.03%
5944
Frontend
Pend 0Skip 0Runs 0
0
0.00%
0.00%
2011
Library
Pend 0Skip 0Runs 0
0
0.00%
0.00%
688

Recent Runs

Recent Daily Suite Runs

DatePipelineStatusSummary
2026-03-14 18:22148497PASSEDTotal 1188 | Passed 1188 | Failed 0
2026-03-15 18:23148499PASSEDTotal 1188 | Passed 1188 | Failed 0
2026-03-16 18:23148604FAILEDTotal 1188 | Passed 1187 | Failed 1
2026-03-17 18:22148823PASSEDTotal 1188 | Passed 1188 | Failed 0
2026-03-18 18:22149034FAILEDTotal 1188 | Passed 1187 | Failed 1
2026-03-19 18:24149182PASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-20 18:22149392PASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-21 18:23149394PASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-14 18:22Pipeline 148497
PASSED
T 1188 | P 1188 | F 0 | Pend 0
2026-03-15 18:23Pipeline 148499
PASSED
T 1188 | P 1188 | F 0 | Pend 0
2026-03-16 18:23Pipeline 148604
FAILED
T 1188 | P 1187 | F 1 | Pend 0
2026-03-17 18:22Pipeline 148823
PASSED
T 1188 | P 1188 | F 0 | Pend 0
2026-03-18 18:22Pipeline 149034
FAILED
T 1188 | P 1187 | F 1 | Pend 0
2026-03-19 18:24Pipeline 149182
PASSED
T 1189 | P 1189 | F 0 | Pend 0
2026-03-20 18:22Pipeline 149392
PASSED
T 1189 | P 1189 | F 0 | Pend 0
2026-03-21 18:23Pipeline 149394
PASSED
T 1189 | P 1189 | F 0 | Pend 0

Recent Smoke Attempts

DatePipelineStatusPassedFailedDuration
2026-03-16 21:17148625PASSED11003m 03s
2026-03-16 21:49148633PASSED11003m 02s
2026-03-16 22:08148635PASSED11002m 58s
2026-03-16 22:38148640PASSED11003m 00s
2026-03-16 22:59148644PASSED11002m 57s
2026-03-16 23:53148646PASSED11003m 00s
2026-03-17 00:06148648PASSED11003m 03s
2026-03-17 00:24148651PASSED11003m 03s
2026-03-17 12:06148704PASSED11003m 04s
2026-03-17 16:09148791PASSED11003m 04s
2026-03-17 16:56148817PASSED11003m 09s
2026-03-18 16:49149024PASSED11003m 10s
2026-03-20 13:16149311FAILED042m 13s
2026-03-20 13:19149311FAILED042m 06s
2026-03-20 13:30149311FAILED042m 13s
2026-03-20 17:03149382FAILED041m 26s
2026-03-20 17:13149382FAILED010m 48s
2026-03-16 21:17Pipeline 148625 job
PASSED
P 110 | F 0 | 3m 03s
2026-03-16 21:49Pipeline 148633 job
PASSED
P 110 | F 0 | 3m 02s
2026-03-16 22:08Pipeline 148635 job
PASSED
P 110 | F 0 | 2m 58s
2026-03-16 22:38Pipeline 148640 job
PASSED
P 110 | F 0 | 3m 00s
2026-03-16 22:59Pipeline 148644 job
PASSED
P 110 | F 0 | 2m 57s
2026-03-16 23:53Pipeline 148646 job
PASSED
P 110 | F 0 | 3m 00s
2026-03-17 00:06Pipeline 148648 job
PASSED
P 110 | F 0 | 3m 03s
2026-03-17 00:24Pipeline 148651 job
PASSED
P 110 | F 0 | 3m 03s
2026-03-17 12:06Pipeline 148704 job
PASSED
P 110 | F 0 | 3m 04s
2026-03-17 16:09Pipeline 148791 job
PASSED
P 110 | F 0 | 3m 04s
2026-03-17 16:56Pipeline 148817 job
PASSED
P 110 | F 0 | 3m 09s
2026-03-18 16:49Pipeline 149024 job
PASSED
P 110 | F 0 | 3m 10s
2026-03-20 13:16Pipeline 149311 job
FAILED
P 0 | F 4 | 2m 13s
2026-03-20 13:19Pipeline 149311 job
FAILED
P 0 | F 4 | 2m 06s
2026-03-20 13:30Pipeline 149311 job
FAILED
P 0 | F 4 | 2m 13s
2026-03-20 17:03Pipeline 149382 job
FAILED
P 0 | F 4 | 1m 26s
2026-03-20 17:13Pipeline 149382 job
FAILED
P 0 | F 1 | 0m 48s
Generated from GitLab project adservio/helm2. Times are shown in Europe/Bucharest. Daily-suite runtime is measured from GitLab pipeline and job timestamps. Category counts come from GitLab test-report JSON artifacts, with job-trace fallback when older artifacts have expired.