Skip to main content

Two ways to know a goal happened

00:03:27

A live sports product is a latency product. If a goal goes in and your app shows it thirty seconds later, you've already lost to the television in the next room and to every app that was faster. So when we built the live-match layer for our football platform, the question was never just "how do we get the data." It was "how do we get it fast, and how do we make sure we never miss it." Those turned out to be two different problems with two different answers, running side by side.

The fast path: a push stream over MQTT

Our data provider publishes live match updates over MQTT. We run one server whose entire job is to hold a long-lived MQTT connection and listen.

php
// Representative: provider topic and host names are genericized.
$mqtt = MQTT::connection();

$mqtt->subscribe('<provider>/football/match/v1', function (string $topic, string $message) {
    foreach (json_decode($message, true) as $item) {
        if (isset($item['score'])) {
            $this->scoreService->process($item['id'], $item['score']);
        }
        if (isset($item['incidents'])) {
            $this->incidentService->process($item['id'], $item['incidents']);
        }
    }
}, 0); // QoS 0 = at most once

$mqtt->loop(true); // blocking, indefinite

A message about a goal is terse: an id, a status code, a couple of numbers. It carries no team names or crests. Fetching those from the database on every message, at the rate live football produces them, is how you cook your primary database. So a scheduled job warms a Redis cache of match metadata every minute, and the consumer reads from there:

php
$match = json_decode(Redis::get("match:{$id}"), true); // warm, no DB hit on the hot path

The hot path never touches MySQL. The slow, batchy work of assembling match metadata happens off to the side on a timer, and the part that runs thousands of times a minute just reads a warm key.

One stream, two kinds of broadcast

The data goes out to clients over websockets, but two surfaces need it differently. The live-scores list, where everyone is watching everything, gets an immediate synchronous broadcast. A single match's detail screen gets its own per-match channel on a queued broadcast.

php
// Firehose for the all-scores list: send now.
class AllScoreEvent implements ShouldBroadcastNow { /* channel: <provider>-football.matchs */ }

// Per-match detail: queued, one channel per match.
class ScoreEvent implements ShouldBroadcast
{
    public $queue = 'websockets';
    public function broadcastOn(): array
    {
        return [new Channel("<provider>-football.match.{$this->matchId}")];
    }
}

Different delivery guarantees for different screens, from the same ingested event. The list has to feel instant; a single match's detail can tolerate riding a queue.

The safety net: a poll every two seconds

Here's the part it's tempting to skip. MQTT at QoS 0 is at-most-once, and our connection uses a clean session, so a disconnect means we miss whatever was published while we were gone, with no backlog to replay. For most data that's an acceptable trade. For "did a goal just happen," missing a message is not acceptable.

So a second, completely independent path polls the provider's REST endpoint every two seconds and reconciles against the same models. The push path gives us speed. The poll gives us a floor on correctness. The two converge on the same broadcast events, so clients can't tell which one delivered a given update.

bash
# the long-lived push worker
php artisan mqtt:football-match-v1

# scheduled on the same role: the reconciling poll + the cache warmer
# match:real-time   -> every two seconds   (REST safety net)
# match:fav-team    -> every minute        (warms the Redis match cache)

The unglamorous half

A long-lived MQTT worker is a process you babysit. Auto-reconnect has to be configured and capped. A clean session means every reconnect re-subscribes from scratch. The worker needs a supervisor to restart it when it dies, because eventually it will. None of that is exotic, but it's the half of "real-time" the tutorials leave out, and it's the half that pages you at 9pm on a match day.

The two-path design is, honestly, an admission that the fast path will sometimes fail. "Bulletproof real-time over a network you don't control" is a fantasy. The realistic version is a fast path that's usually right and a cheap, dumb poll that catches what it drops. Two ways to know a goal happened, because one way is never quite enough.