Marcus Povey

In the previous post I argued, fairly bluntly, that if we want EOSC catalogue federation to become real, we need to stop treating it as a purely conceptual architecture problem and start pointing code at actual endpoints.

The first hackathon proved the basic point. We could take service catalogue metadata from one node, discover it, parse it, map it into a DCAT-compatible representation, and make it useful somewhere else. Not production infrastructure, or a final standard, but enough to move the conversation from “could this be done?” to “here is the thing we made, now tell us where it breaks.”

The second hackathon was useful because it broke in more interesting ways.

The goal was still deliberately small

I am becoming increasingly convinced that the only survivable way to build federation is in small, working pieces.

So, the scope for this session was intentionally narrow again: discover node capabilities, find service catalogue feeds, pull enough metadata for discovery, and test mappings between the formats already circulating in the EOSC ecosystem. In practice this meant looking at the relationship between EEN, DCAT / DCAT-JSON-LD, and the EOSC Beyond service catalogue and node capability work.

That last part matters.

After the first hackathon, we had shown a plausible path between the Life Sciences side and Data Terra using DCAT-compatible catalogue exchange. But EOSC is not made of two nodes, and it is certainly not made of one preferred implementation. If federation is going to work, it has to survive contact with other projects, other catalogues, other assumptions, and other software.

So this time the room included people from Data Terra, EOSC Beyond, EGI, GRNET, EMBRC, and the Life Science Connect node. Which is another way of saying: enough different perspectives to stop the prototype from getting too comfortable.

Discovery is not just a feed URL

One of the first useful discussions was around node capability discovery.

In my earlier work, I had been experimenting with a simple .well-known endpoint as a way for a node to say, effectively: “these are the catalogue and capability endpoints I expose.” It is a deliberately small idea, and therefore exactly the sort of thing one can implement quickly and argue about usefully.

The EOSC Beyond work approaches the same problem through its Node Registry and capability model. A node can advertise capabilities such as resource catalogue, service catalogue, monitoring, helpdesk, AI, and so on, with endpoint URLs, protocol, app spec, version, and operational status.

That is richer than “here is my feed”, but also makes the shape of the future federation problem clearer.

If EOSC nodes are going to interoperate peer-to-peer, they need some way to discover what each other can do. Not just “do you have a catalogue?”, but “what kind of catalogue?”, “which API does it speak?”, “what version?”, “is it operational?”, and eventually “how do I authenticate against it?”

We deliberately left authentication out of scope for this hackathon, because otherwise we would have spent the whole session digging a hole and admiring the sides. But it is now much clearer where it needs to fit.

The good news: the bridge exists

The practical win from my side was that I was able to interrogate an EOSC Beyond service capability endpoint, find the catalogue, and map service data through an EOSC Beyond to EEN to DCAT path.

Was it lossless? No.

Was it elegant? Also no.

Was the actual service entry preserved well enough to prove the bridge is viable? Yes.

That is the important bit. We now have evidence that an EOSC Beyond-facing catalogue can be connected to the DCAT-compatible federation work without first requiring everyone to stop, redesign everything, and agree a perfect common model in advance.

This is exactly why building small things helps. A whiteboard version of this discussion can easily turn into a debate about which model is more correct, however a working version forces a better question: what information survives the mapping, what is lost, and does the loss matter for the use case in front of us?

For simple discovery, the answer looks encouraging. For richer service management, ordering, monitoring, access policies, and deployment, we will need more explicit profiles and extensions.

But that is a much better problem to have.

Data Terra brought the other half of the proof

Data Terra showed something equally important from the other direction.

Their catalogue uses a rich DCAT-based pivot model internally, covering resources such as datasets, software, publications, and services. From that model they can expose different formats, including EEN, using API parameters. In other words, they are not treating EEN and DCAT as mutually exclusive religious positions. They have a richer local model and generate the exchange format needed for a given interaction. That feels like the right shape.

Even better, they demonstrated federated catalogue results combining Data Terra, the Life Science node, and ENVRI services. ENVRI was not plug-and-play, even though it also uses DCAT, because the profile and serialisation differ. But that is exactly the sort of implementation detail we need to discover early.

“Uses DCAT” is not enough.

That was one of the clearest lessons of the session. DCAT is useful precisely because it gives us a shared language, but it is flexible enough that two groups can use it differently and still fail to interoperate cleanly. If we want federation to work, we need a shared EOSC service profile, not just a shared fondness for the same acronym.

The less comfortable news: semantics will not go away

One contributor made an important point that I think is easy for technical people, myself included, to underplay: some of this is not a mapping problem. It is a meaning problem.

Research infrastructures expose different kinds of “services”. Some are infrastructure-level services. Some are data access services. Some are physical access services. Some are catalogues, platforms, libraries, pipelines, facilities, or bundles of several things that only look simple from far enough away.

You can map fields all day and still not answer the more important question: are we describing the same kind of thing?

That does not mean we should stop the technical work until the semantics are perfect, that way lies another six months of meetings and a very tidy shared document. But it does mean the technical profile needs to leave room for service typology, provenance, and domain-specific extensions.

This is where I think the “minimal base plus extensions” model becomes important.

At the base level, every node should be able to expose enough information for discovery: name, description, landing page, provider, identifiers, source, and so on. Then, where a node supports richer capabilities, it should be able to advertise those explicitly. EOSC Beyond extensions, ordering extensions, monitoring extensions, data access extensions, whatever proves necessary.

The trick is to keep the base small enough that people can implement it, but precise enough that implementations do not drift apart immediately.

What we learned this time

The first hackathon showed that catalogue federation could work between real participants, the second showed that the path is broader than one mapping and messier than one format.

We learned that EOSC Beyond’s Node Registry and capability model is addressing a real need: nodes need to advertise what they can do in a machine-readable way. We learned that DCAT can work as a useful pivot, but only if we define a clear EOSC service profile and serialisation expectations. We learned that EEN remains unavoidable as the current minimum operational exchange format, but that it should not be the ceiling of what a federation can express.

We also learned that a one-way bridge is not enough. Mapping EOSC Beyond to EEN/DCAT is useful, but the reverse direction matters too. If these ecosystems are going to converge rather than quietly diverge, we need to write down the crosswalks, compare what is lost, and decide which differences are acceptable.

That is the next piece of work.

The point, again

There is still plenty to discuss. We need a formal capability descriptor spec, or at least something close enough that different nodes can implement it. We need a clearer DCAT service profile, we need reverse mappings, and we need to understand how federated search behaves across heterogeneous catalogues. We need to bring the semantic service typology discussion into the same room as the API discussion before the two drift too far apart.

But the important thing is that these are now grounded problems.

They came from code touching real endpoints, not from abstract anxiety. That changes the nature of the work, it gives us examples, it gives us failure modes, and it gives us something to improve.

This is why I keep coming back to the same point: build something small, then let reality correct it.

After the first hackathon, EOSC catalogue federation was no longer just theoretically possible. After the second, it is starting to look like an engineering programme: a small base profile, explicit capabilities, practical mappings, known semantic gaps, and a growing set of real nodes to test against.

That is progress. Not the grand unveiling of federation as a finished thing, but something better: federation becoming less imaginary, one endpoint at a time.