Engineering Council Test Reliability Report

Scope aligned with Slack channel #dezvoltare, covering 2026-03-21 07:00 to 2026-03-28 07:00. Metrics and timings are sourced from GitLab pipelines, jobs, and test-report artifacts for the daily 6 PM regression suite and the production smoke suite. Trend charts use daily buckets across this window.

Executive Snapshot

7
Daily Runs
3/7
Daily Green
19m 28s
Avg Daily Runtime
13
Smoke Attempts
11/13
Smoke Green
3m 14s
Avg Smoke Runtime
2m 56s
Median Smoke Time
0
Current Green Streak

Council Narrative

  • Daily suite reliability finished strong: the last 0 daily runs were all green.
  • In the last 10 daily runs, total test volume grew from 1,189 to 1,189.
  • Library has the highest strict failure ratio at 1.00%, while Billing has the broadest non-pass footprint at 0.00% and showed up in 0 failed runs.
  • The average daily-suite runtime, measured from GitLab start and finish timestamps, was 19m 28s.
  • Smoke runs stayed fast when healthy: average duration was 3m 14s and the median passing run was 2m 56s.
  • Across the latest daily smoke references, observed test volume moved from 110 to 10.
  • Smoke-suite breakdown in this window: Frontend 5/7, University 6/6.
  • 2 production pipelines clearly recovered on rerun after an initial smoke failure: 149634, 149671.

Category Failure Ratios

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

Category Failure RatioBilling0.00%Web0.00%Frontend0.23%Library1.00%
Category Non-pass Ratio (fail + pending + skipped)Billing0.00%Web0.00%Frontend0.23%Library1.00%
Daily Daily Suite Status0000103-2103-2303-2503-27
Daily Smoke Attempts0135703-2103-2303-2503-27
Daily Average Daily Suite Runtime19m 07s19m 18s19m 29s19m 40s19m 51s03-2103-2303-2503-27
Daily Average Smoke Runtime0m 00s0m 53s1m 46s2m 40s3m 33s03-2103-2303-2503-27
Daily Suite Total Test Growth (Recent 7 Runs)1189118911891189119003-2103-2303-2503-27
Smoke Suite Total Test Growth (Latest Run Per Day)
FrontendUniversity
10356085110Frontend 03-23: 110Frontend 03-24: 110Frontend 03-25: 110Frontend 03-27: 110University 03-23: 10University 03-24: 10University 03-25: 10University 03-27: 1003-2303-2403-2503-27

Category Aggregate Table

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

How computed

Failure Ratio = failed test executions divided by total test executions in the selected timeframe.

Non-pass Ratio = (failed + pending + skipped) test executions divided by total test executions in the selected timeframe.

Example: a 1.75% Billing failure ratio means 1.75% of all Billing test executions in this period ended in failed. A 1.35% Library non-pass ratio means 1.35% of all Library test executions in this period ended in failed, pending, or skipped.

CategoryTotalFailedPendingSkippedFailure RatioNon-pass RatioRuns With Failures
Billing7560000.00%0.00%0
Web52010000.00%0.00%0
Frontend17644000.23%0.23%3
Library6026001.00%1.00%3
CatFailF%NP%Tot
Billing
Pend 0Skip 0Runs 0
0
0.00%
0.00%
756
Web
Pend 0Skip 0Runs 0
0
0.00%
0.00%
5201
Frontend
Pend 0Skip 0Runs 3
4
0.23%
0.23%
1764
Library
Pend 0Skip 0Runs 3
6
1.00%
1.00%
602

Recent Runs

Recent Daily Suite Runs

DatePipelineSuitesStatusSummary
2026-03-21 18:23149394BillingWebFrontendLibraryPASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-22 18:23149456BillingWebFrontendLibraryPASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-23 18:23149694BillingWebFrontendLibraryFAILEDTotal 1189 | Passed 1185 | Failed 4
2026-03-24 18:22149866BillingWebFrontendLibraryFAILEDTotal 1189 | Passed 1186 | Failed 3
2026-03-25 18:22150059BillingWebFrontendLibraryFAILEDTotal 1189 | Passed 1187 | Failed 2
2026-03-26 18:22150180BillingWebFrontendLibraryPASSEDTotal 1189 | Passed 1189 | Failed 0
2026-03-27 18:22150328BillingWebFrontendLibraryFAILEDTotal 1189 | Passed 1188 | Failed 1
2026-03-21 18:23Pipeline 149394BillingWebFrontendLibrary
PASSED
T 1189 | P 1189 | F 0 | Pend 0
2026-03-22 18:23Pipeline 149456BillingWebFrontendLibrary
PASSED
T 1189 | P 1189 | F 0 | Pend 0
2026-03-23 18:23Pipeline 149694BillingWebFrontendLibrary
FAILED
T 1189 | P 1185 | F 4 | Pend 0
2026-03-24 18:22Pipeline 149866BillingWebFrontendLibrary
FAILED
T 1189 | P 1186 | F 3 | Pend 0
2026-03-25 18:22Pipeline 150059BillingWebFrontendLibrary
FAILED
T 1189 | P 1187 | F 2 | Pend 0
2026-03-26 18:22Pipeline 150180BillingWebFrontendLibrary
PASSED
T 1189 | P 1189 | F 0 | Pend 0
2026-03-27 18:22Pipeline 150328BillingWebFrontendLibrary
FAILED
T 1189 | P 1188 | F 1 | Pend 0

Recent Smoke Attempts

DateSuitePipelineJobStatusPassedFailedDuration
2026-03-23 12:20Frontend149520Frontend smokePASSED11003m 08s
2026-03-23 15:20University149634University smokePASSED1002m 09s
2026-03-23 15:25Frontend149634Frontend smokeFAILED10375m 41s
2026-03-23 16:25University149671University smokePASSED1002m 17s
2026-03-23 16:31Frontend149671Frontend smokeFAILED10375m 57s
2026-03-23 17:03University149684University smokePASSED1002m 33s
2026-03-23 17:05Frontend149684Frontend smokePASSED11003m 03s
2026-03-24 20:52University149788University smokePASSED1002m 56s
2026-03-24 20:53Frontend149788Frontend smokePASSED11003m 10s
2026-03-25 12:14University149902University smokePASSED1002m 23s
2026-03-25 12:16Frontend149902Frontend smokePASSED11003m 05s
2026-03-27 14:33University150306University smokePASSED1002m 23s
2026-03-27 14:35Frontend150306Frontend smokePASSED11003m 15s

Smoke Suite Breakdown

Frontend
7 attempts across 7 pipelines
71% green
Passed5
Failed2
Incomplete0
Avg runtime3m 54s
Median passing runtime3m 08s
Pipelines7
University
6 attempts across 6 pipelines
100% green
Passed6
Failed0
Incomplete0
Avg runtime2m 27s
Median passing runtime2m 23s
Pipelines6
Generated from GitLab project adservio/helm2. Times are shown in Europe/Bucharest. Daily-suite runtime is measured from GitLab pipeline and job timestamps. Category counts come from GitLab test-report JSON artifacts, with job-trace fallback when older artifacts have expired.