Tunnel: Internal Data Flow Over the Internet

Overview

Tunnel allows incoming traffic from services outside of our AWS private network to access that network, generally in order to communicate with some other service that runs in ECS and isn't accessible over the internet itself.

An example use case for this is Stripe. We need to process Stripe events in the Wallstreet service, but don't want to expose Wallstreet to the internet via HTTP just so it can acknowledge webhooks. Besides, maybe another service beyond Wallstreet is interested in those same Stripe events.

API

v1 HTTP Routes

The base url is tnl.pivot.app/. Each route defines its own authentication requirements and route versioning.

POST /stripe/v1/hook

Stripe events are webhooks that are needed by Wallstreet. Tunnel's job is simply to pass them along by converting them from HTTP to NATS messages, so that a quick 200 response can be provided by Tunnel to Stripe, simply acknowledging receipt, after which Wallstreet can process them asynchronously.

POST /mux/v1/hook

Mux fires events related to media files which are needed by Blobby. Like with Stripe, Tunnel simply converts these to NATS messages.

POST /livekit/v1/event

LiveKit must be configured to send WebhookEvent messages of type participant_joined and participant_left to support Stagehand use cases. Tunnel simply converts these to NATS messages.

GET/POST/PATCH admin/v1/**

PivotAdmin needs to query and mutate specific data in specific ways. Tunnel provides an HTTP API (using the same Connect protocol that Friend does) and wrapping other services.

Authentication of admin Routes

We delegate AuthN and AuthZ of the actual Pivot Technologies team members using PivotAdmin to the PivotAdmin application server. Tunnel only needs to authentication that the PivotAdmin application server is who it says it is. To accomplish this, we use two layers:

1. Before forwarding traffic to the Tunnel application container, the Aplication Load Balancer verifies that the source IP of the traffic is in the IPv6 CIDR reserved to the PivotAdmin VPC.
2. Tunnel expects a Signature header. Tunnel uses a public key it has to verify that PivotAdmin signed this exact request body with the corresponding private key which only exists in the PivotAdmin AWS account, in Secrets Manager.

Therefore, both PivotAdmin VPC access and access to the private key would be needed to write malicious requests to this API, and private key never leaves the PivotAdmin account.

Databases

N/A

NATS

Publication

• tunnel.incoming_webhook_events.* - for Stripe, Mux, LiveKit events. (.stripe, .mux, .livekit)

Consumption

N/A

Temporal Workflows

N/A

Deployment

Security

• Rate limiting at the firewall level.