EMM@: when AI isn't intelligent...
and why.

EMM@ is the AI model developed with the declared intent of asserting Italian technological sovereignty in AI — and which concretely revealed the complexities of AI implementation in the public sector.

What follows is a satirical simulation of EMM@ — not the original service. We trained a model to respond like EMM@ to illustrate what many have already experienced.

You have 5 free questions. Use them wisely.

The test

We asked the simplest questions we could imagine. No tricks, no technical questions. Things that anyone who attended primary school would know.

You've just seen the results. Or you will, if you still have questions to use.

It's tempting to stop at the punchline. But there's something worth understanding: why does this happen? Why does a system built to answer citizens' questions fail on the simplest ones?

The answer isn't "it's just a bad product." It's more structural than that.

Why AIs fail

A language model in its base form doesn't "know" things the way a human does. It doesn't have an internal representation of reality: it has statistical patterns extracted from billions of texts. When it answers, it's not retrieving a verified fact — it's generating the most probable sequence of words given the context. It's a distinction that seems technical but changes everything in practice.

This works surprisingly well for many things. And fails spectacularly on others — often precisely the simplest ones, where the correct answer is so obvious that the model encountered millions of variations during training, including wrong answers, rhetorical questions, ironies. The model learns the pattern of the elaborate answer, not necessarily the correct one.

But — and this is the point worth understanding — this only describes the behavior of a model used alone, without additional layers. A well-designed system doesn't ask the model to "remember" things. It provides them at the moment they're needed.

The hallucination problem

"Hallucination" refers to the phenomenon where a language model produces plausible but false information — dates, names, citations, laws — with the same confidence it uses when stating real facts.

It's not a bug in the conventional sense. It's a structural feature of models trained to maximize text coherence, not factual accuracy. The model has no internal mechanism that distinguishes "I know this" from "I'm generating something plausible."

For a general-purpose assistant it's an acceptable limitation. For a public service that answers citizens' questions about taxes, deadlines, regulations — it's a serious problem.

Sources, training and updated context

The second problem is context. Large models have a cutoff date — a date beyond which they haven't seen data. A model trained in 2023 knows nothing about what happened in 2024 or 2025, unless it's provided with the context at the time of the request.

For a company that wants to use AI on internal data — price lists, procedures, products, industry regulations — this is the central issue: the base model doesn't know your company. It doesn't know your products, your processes, your clients.

There are two ways to address this. Fine-tuning — training the model on your own data — is expensive, slow, and produces results that become outdated as soon as the data changes. RAG (Retrieval-Augmented Generation) retrieves relevant documents at the moment of the question and provides them to the model as context. It's more flexible, updatable, and for most business use cases it's the right choice.

Our approach: layered memory

The Knowledge Base we build for each client works on a principle opposite to that of the base model: instead of asking the model to remember, we provide it with the right information at the moment it's needed. The client's documents are indexed and when a question arrives, the system retrieves the relevant passages and passes them to the model as real context. The model doesn't invent — it reads.

The result is an agent that responds based on what's in the Knowledge Base, not on what it statistically learned during training. If the answer isn't in the documents, the agent says so instead of constructing a plausible one.

But document retrieval is only the first layer. A system designed to last over time adds something more: a structured representation of the client's knowledge — not just the raw documents, but the concepts, the relationships between them, the shared definitions. This structure is built and maintained over time, not generated on the fly for each question. It's the difference between having an archive and having an organized library.

The third layer is interaction memory. A system that doesn't remember previous conversations starts from scratch every time — it doesn't learn from already-handled cases, doesn't adapt to the client's context over time, doesn't accumulate operational experience. Integrating persistent episodic memory transforms the agent from a response tool to a collaborator that grows with the company.

These three levels — document retrieval, structured knowledge, experience memory — aren't additional features. They're architectural choices made at the beginning or not at all. They can't be added to a system already in production without starting over.

EMM@ doesn't have them. Not because its builders didn't think about it — but because building this way requires time, cost, and a clear vision of what the system should do in the long term. A rapid prototype by definition doesn't have that.

Frequently asked questions