Devo

2025

Abstract

Group CI failures, explain likely causes, and keep a history of fixes so debugging starts with context.

1   Background and motivation

A failed CI run often sends a developer back through the same investigation: locate the relevant log lines, identify the failing step, and search for a similar incident. Devo was built to retain that context across failures. Its aim is to make prior evidence available at the start of an investigation rather than require each red build to be interpreted from scratch.

2   Ingestion and representation

The ingestion layer extracts failure information from CI logs and organizes it around repositories, workflows, runs, and errors. A knowledge graph represents relationships between those entities, while text vectors make similar error messages searchable. The documented implementation uses TF-IDF representations for this similarity layer, so the retrieval mechanism is more specific than a generic claim of semantic understanding.

Graph traversal and text matching provide complementary context. Similar wording can connect failures whose logs look alike, while graph relationships establish where those failures occurred. Combining them gives the analysis stage a set of relevant prior records rather than only the current error string.

3   Analysis workflow

The FastAPI service exposes separate operations for ingestion, graph construction, failure analysis, and querying related incidents. The web interface lets a developer inspect a run, search previous failures, and view system statistics. Optional language-model diagnosis uses the retrieved context to explain likely causes and suggest an investigation path.

This arrangement keeps evidence collection separate from the generated explanation. A useful diagnosis should direct attention to the failing step and related records, leaving the developer able to check whether a suggested cause fits the actual run. The history also supports looking for recurring failure patterns instead of treating every event as isolated.

4   Interpretation and limits

Similar logs do not necessarily share a root cause: an infrastructure timeout and an application bug may surface through the same final error. Devo’s explanations are therefore investigation aids, not confirmed fixes. The quality of the result depends on what logs were ingested, how consistently failures were represented, and whether the retrieved examples are relevant to the current change.