Peter Marshall: Imply: logs, compression, agents

Why this conversation

Agents read logs the way no human does: everywhere, at once, and all night. Peter Marshall is Director of Developer Relations at Imply, and had just given a talk about watching one do it. The host recorded the conversation on 2 November 2026 at the Imply stand at SREday Amsterdam.

Imply builds for one job: “we have focused everything we built on the compute and the storage of telemetry data, anything observability.” And “we’re focused particularly at the moment on logs. So anything where somebody needs to get very fast responses or store lots of log data, that’s where we come in.”

A bank that did not know what it needed

The biggest use case lately? “Recently, it has definitely been the investigation of lots of cybersecurity log data.” A bank Imply works with had an issue and needed to get at data in a huge area very quickly. What it faced, “as many people have to do with observability platforms, is to rehydrate the whole lot. They didn’t know which bits they needed.” Nor which period of time or which field. “So we helped them to just make it available in our storage layer.” Nowadays, Peter added, Imply does it in the customer’s own storage, and brings the compute power to search it quickly.

Compression, and what it buys

“Our compression technique is amazing.” It is a new compression algorithm: “This compression algorithm is like at least two, sometimes 20.” Some of Peter’s colleagues on the stand “have been in POCs where you’ve got 100 times better compression.” The point is not the disk alone: “So you’re getting much better performance and you’re getting cheaper storage.” Think of the dashboard with parts switched off because they take too long, or the saved searches that consume a whole cluster overnight: better indexing and better compression mean they run.

There are two ways to use it. Ingest the data and give it its indexing layer, “or you can leave it in a bucket, an object store, and you build just the index layer over the top of it.” And there is no schema to write first: “We do something behind the scenes that is structure aware.” A schema still goes on top, to make the data available to people, but the compression works at a lower level.

Let loose: an agent on the logs

For the talk, “we just set up a very simple, basic agent.” It looked at synthetic data that faked an attack, using a library of pre-built searches, while a second agent ran the investigation across many indexes and kinds of log data. Once it ran, “it became non-deterministic, the patterns are all over the place.” Peter’s advice is in the middle of a sentence: “monitor your agents”. And the picture of what it was like: “it’s like you’ve let a teenager loose on this job and it does it really well”. The platform under that has to cope with the speed and the volume: “And that’s why we have Lumi in the background for those agentic workflows”.

No agent has to be installed on your machine. “we’ve just got a simple MCP, an endpoint”, and Peter uses it himself: “I use it all the time through Claude Code to check the code I’ve written by looking at the logs I’ve stored in Lumi”. For the demo, Peter gave Claude Code the goal, the endpoint and some sample data. It took iteration, and that is where Peter is strict: “the need for you to test agents’ outputs has never been more critical.” The answers are not yes or no: “98% of the time it was about right.” Five or ten seconds a search is fine, but “if you could do it and your responses come back in half a second when you’re doing these iterations”, that is a lot of time saved when agents are busy all the time. How much speed you pay for is your choice, and what to keep hot.

Getting started

“So, best thing to do is go to imply.io.” And for engineers: “If you’re a technical person, I can highly recommend the documentation.” It walks through how things work, from deploying with Terraform to the queries you can write; then book a demo with one of the Imply team.

When it comes out

The episode is edited and in the release order. Like every episode it goes out on YouTube first, then on Apple Podcasts, Spotify and Amazon Music.