10 · ObservabilityLab 51 / 59
Challengelogging
Centralised Logging with Loki and Grafana — Challenge
Ship every Pod's logs somewhere they survive the Pod, then answer a real question with them.
- Time
- 25 min
- Level
- Intermediate
- Objectives
- 4 objectives
- Cost
- Free
Where this fits in the platform
Already built
This lab adds
- Logs you can search after the pod that wrote them is gone
Which lets you
—
Before you start
You will need
- kind or minikube
- Helm 3.14+
- kubectl
You do not need these already — the lab environment below provides them.
You will be able to
- Query logs by label instead of grepping a stream
- Correlate a metric spike with the log lines behind it
- Explain why Loki indexes labels rather than content
Cost — Free
on kind or minikube. On a cloud cluster Loki requests a persistent volume, billed per GB-month — see cleanup.
The goal#
Achieve the same outcome as Centralised Logging with Loki and Grafana, from an empty starting point, without the steps.
A Pod crashed at 3am. It has been replaced, and kubectl logs shows the new one. The evidence went with the old container.
Metrics told you that something broke. Logs are how you find out why — but only if they left the node before the Pod did.
Hands-on environment
Run this lab in a real terminal, free and in your browser. The environment is temporary and yours alone — break it as much as you like.
Start the challengeOpens in Killercoda, in a new tab — keep this page open for the steps.
This is the guided lab's environment — the same machine, with its walkthrough on the left. Work from the task above and leave those steps alone until you are done, or you are reading the answers.
Run it on your own machine
Run this lab on your own machine. One command starts the environment, with everything the lab needs already installed:
You will need:
- docker
- kubectl
- kind
git clone https://github.com/EgyKode/EgyKode-lab.git
cd EgyKode-lab
./egykode start k8s
./egykode shellYou need Docker and Git installed. Everything else runs inside the environment. The first start downloads it and takes a few minutes; later starts are seconds.
Not sure what you already have? Run: npm run doctor — it checks and changes nothing.
Anything you tick here is your own record. EgyKode cannot see inside that terminal, so the success criteria stay self-assessed even when the environment checks your work for you.
What must be true when you are done
Step 1 of 3
What must be true when you are done#
- Logs from every namespace are queryable in Grafana.
- You can filter to one Deployment's errors in the last 15 minutes.
- You produced a crash and found its cause from the logs alone.
- You can explain what happens to logs when a Pod is deleted, with and without Loki.
Rules#
- Do not open the guided lab until you are finished, or until the same problem has held you up for 20 minutes.
- Documentation is allowed and encouraged.
- Verify every criterion with a command whose output you can read.
If you get stuck#
- What did you expect, exactly?
- What happened instead — the error text, not a paraphrase?
- Which layer is that error from?
- What is the smallest command that proves the layer below is fine?
You are done when
0 of 4
The concept behind it
Phase complete · 10 Observability
You can now: Metrics, logs, dashboards and alerts exist — and the alerts are ones a human can act on.
Next phase
Lab 51 of 59 on the project path