This case study presents an external project developed by Recursos en la Red for SPORT (Grupo Prensa Ibérica). It is included in the CARL Library as an example of data-driven innovation in the media industry and was not developed as part of the CARL project.
In a digital newsroom, speed can never be separated from rigor. SPORT works every day in a news environment shaped by constant updates, the coexistence of different formats, and the need to provide readers with context in increasingly shorter timeframes.
The MENTOR project, developed by Ediciones Deportivas Catalanas, S.A.U., analyzes how to integrate intelligent capabilities into the digital CMS to facilitate tasks within the newsroom workflow, always keeping professional judgment as the guiding reference.
Use case on the role of data in the development, evaluation, and continuous improvement of an intelligent assistant within the digital CMS
In a digital newsroom, incorporating artificial intelligence is not simply a matter of adding a new button to the working environment. The real challenge is to understand which editorial tasks can benefit from assisted automation, how to measure their impact, and how to ensure that every suggestion generated by the system maintains the quality, judgment, and human supervision required by journalistic production.
The Challenge
Newsrooms already work with AI tools, but often in a fragmented way, outside the editorial system and without common criteria for use, quality, or security.
MENTOR addresses this problem by integrating intelligent assistance capabilities into the digital CMS: headline and subheadline suggestions, SEO variations, social media copy, style correction, translation, transcription, summary generation, semantic retrieval, and support for content based on verified sources.
For these functionalities to deliver real value, data analysis must be involved before the tool is built, during its launch, and after its adoption by the newsroom.
The Role of Data Analysis
1. Analyzing the Corpus Before Designing the AI
The starting point is to study the corpus of documents and editorial content on which the system will operate. Not all texts, sections, formats, or sources involve the same level of complexity.
Exploratory analysis makes it possible to detect style patterns, content typologies, translation needs, recurring transcription requirements, the use of background information, and differences between pieces that require light intervention and pieces that demand more intensive assistance.
This work helps define a useful configuration of instructions, editorial guidelines, and parameters for each use case, preventing the AI from functioning as a generic tool disconnected from the newsroom’s actual practices.
2. Sizing the Problem and the Value of Each Intervention
Data analysis makes it possible to answer an essential question: how often it makes sense for AI to intervene in the editorial workflow, and what impact each intervention has.
In MENTOR, generating a headline suggestion, translating a text, summarizing a press conference, or retrieving background information through semantic indexing do not involve the same cost or provide the same value. Each function requires measuring usage frequency, time saved, expected quality, inference cost, and editorial risk.
This sizing helps decide when to use faster and more economical models, when to reserve more capable models for more complex tasks, and how to balance quality, cost, and response time within the digital CMS.
3. Simulating Scenarios Before Deployment
Before taking a functionality into production, analysis based on historical data makes it possible to simulate scenarios: types of news stories, usage volumes, simultaneous requests, depth of semantic search, languages, formats, and edge cases.
These simulations make it possible to check whether the response arrives in time for the pace of editorial work, whether the system maintains quality as load increases, and whether the results remain useful for complex, urgent, or sensitive content.
The goal is not always to maximize AI intervention, but to find the point at which assistance improves the process without introducing friction, unnecessary cost, or risk of losing editorial control.
4. Evaluating the Real Quality of the Output
The quality of an AI functionality cannot be measured solely by whether it produces a grammatically correct response. In a newsroom, the output must be useful, verifiable, consistent with the outlet’s style, and appropriate to the news context.
MENTOR needs to combine editorial review, usage metrics, and AI evaluations to analyze whether headline suggestions, summaries, translations, transcriptions, or semantic retrieval results fulfill their purpose.
A complementary technique is the use of LLM as a judge: a more powerful model evaluates representative samples of results and helps generate comparable quality metrics, always as support for human evaluation and not as a replacement for editorial judgment.
5. Observing the System Once It Is Running
When the tool is deployed, another phase of analysis begins: measuring the real adoption of the functionalities. A functionality may be technically available and still fail to become naturally integrated into journalists’ routines.
Observability makes it possible to analyze which functions are used, at what points in the editorial workflow, with what acceptance rate, how often the suggestion is edited, and which interaction patterns indicate satisfaction, rejection, or unexpected use.
Drops in usage may warn of a degradation in response quality, excessive latency, poorly calibrated instructions, or a user experience that does not fit the newsroom’s actual way of working.
Measurement Map for the Use Case
| Analyzed Dimension | What It Makes It Possible to Measure | Decision It Helps Facilitate |
| Editorial corpus | Detect style patterns, formats, languages, and contextual needs. | Configuration of guidelines, prompts, and semantic retrieval. |
| Cost and latency | Compare models, response times, and the complexity of each task. | Selection of the right model for each intervention. |
| Output quality | Measure usefulness, accuracy, coherence, subsequent editing, and acceptance. | AI evaluations, human review, and iterative improvement. |
| Real usage | Observe adoption, frequency, abandonment, feedback, and unforeseen interactions. | Product adjustments, training, and new functionalities. |
Transferable Learning
The main learning from the case is that an artificial intelligence application in media must be managed as a measurable system. Quality cannot be assumed: it is observed, evaluated, compared, and corrected. Adoption cannot be taken for granted either: it is analyzed based on users’ actual behavior, their feedback, and the usefulness the tool demonstrates within the workflow.
This approach allows AI to evolve with the product. Historical data helps design the first version; usage data helps adjust the experience; quality evaluations help calibrate models and instructions; and observability makes it possible to detect degradation, improvement opportunities, or new use cases.
Conclusion
MENTOR turns data analysis into a governance layer for the AI product. Data helps decide what to automate, how to do it, how much to invest in each intervention, and when to change course. In this way, artificial intelligence is integrated into the digital CMS as measurable and supervised assistance, aimed at improving the productivity and quality of editorial work without displacing professional judgment.
