Local Development in the Pivot Repositories

Clone `pivot` and `pivot-internal`

Install system dependencies

Git
Docker
Node and npm
pnpm
Go
Rust/Cargo (for Tauri desktop app builds)
Terraform CLI (if you will be editing Terraform modules in pivot-internal)
Atlas CLI (opens in a new tab)

`pnpm install`

From the root of each repository.

`tools/scripts/go-tools-install.sh`

From the root of the pivot repo to install Go CLI dependencies, such as for generating Protobuf code. You likely need to chmod +x tools/scripts/go-tools-install.sh before running this shell script.

`npx buf generate`

This is a necessary codegen step whenever cloning the repo or changing .proto files. It will generate a folder called gen/ that is ignored by Git.

Use `docker compose`

Running docker compose up from the root of the pivot repository starts all backing services like Postgres, Cassandra, Valkey, NATS, and AWS service emulations (S3, SES). Turbopuffer is a managed service and cannot be run locally; development involving Quest may require mocking or connecting to a shared dev instance.

Once the necessary backing services are started, all our container-based backend services (that we deploy to ECS in staging/production) will start as well (and should automatically apply any Postgres database migrations). Docker will watch the source directories for those services and will rebuild the image and redeploy into the docker compose environment when changes are made.

As described here, Docker will inject configuration values and secrets using an entirely local stack, so no real secrets are needed unless you need to connect to a dependency that can't be run or emulated locally. If so, these overwridden values can be set using the --env-file CLI argument (opens in a new tab).

Alternatives

AMD64

The following commands have been tested on an AMD64 architecture and serve as alternatives to running docker compose up directly.

docker compose up database nats nats-init localstack localstack-init will run create (if not existent) and start containers for Postgres, NATS, and LocalStack.
(npx or pnpm) nx serve {projectName} can be used to run individual projects (only one project at a time).
(npx or pnpm) nx run-many -t serve -p {projectName1} {projectName2}... can be used to run multiple projects at once.

Seeder

Run the seeder app with pnpm nx execute seeder to add data to the running Docker services. We also use Seeder in CI to test backend service APIs for breaking changes.

Use `nx`

Both pivot and pivot-internal use Nx as a multi-language monorepo framework. pnpm nx is used to serve, build, test, and lint. Prefix your nx commands with pnpm rather than installing Nx globablly and rather than using npx to ensure that all developers use the version of Nx specified in the package.json.

Run pnpm nx serve {projectName} with the name of the project from the relevant project.json file to run apps in development mode. For the pivot repo, you only need to do this after running docker compose up if you want to run frontend apps. The backend services that run in ECS in production are all started via Docker (though you can run serve for them as well if you want one outside of Docker -- hot reload will be faster that way).

As described here, for applications that require environment variables, Nx will inject them based on the .env file in each app directory, which is committed to source control. You can create a .env.local file to override any variable defined in the .env with actual secrets that will be gitignore'd.

Expo Custom Development Client

For previewing frontend changes on mobile, you will need to install our Expo custom development client -- Expo Go will not work because we have custom native modules beyond the Expo standard runtime.

Using NATS

You can manually explore NATS streams and subjects with the NATS CLI.

nats sub --user pivot_facebox_user --password pivot_facebox_password facebox.change_feed.user.updatedV1

Installing Dependencies and Creating New Services Using Make

Check out the MAKEFILE to explore the available commands.

Generating a new Go backend service:

make cookiecutter
make service SERVICE_NAME=<<service_name>>

Using Postgres, Ent, Atlas

For backend Go services that use the Ent ORM, Atlas is used for migrations against Postgres.

Ent

Create entities

# from apps/facebox
 
go run -mod=readonly entgo.io/ent/cmd/ent new EntityName

Generate code

# from repo root
 
go generate ./apps/facebox/internal/ent

Atlas

Note that the Atlas CLI must be installed on your development machine to create (or even apply) migrations. This means the app will fail to start without first installing the Atlas CLI (opens in a new tab).

Create a new migration

cd apps/facebox
 
atlas migrate diff migration_name \
  --dir "file://internal/ent/migrate/migrations" \
  --to "ent://internal/ent/schema" \
  --dev-url "docker://postgres/16/test?search_path=public"

Update your local database to match the Ent schema (good for local dev before you actually create a migration)

atlas schema apply \
    -url "postgres://pivot_facebox_user:pivot_facebox_password@localhost:5432/pivot_facebox?sslmode=disable" \
    --to "ent://apps/facebox/ent/schema"

Wipe your local database (if schema apply is failing)

atlas schema clean \
    -url "postgres://pivot_facebox_user:pivot_facebox_password@localhost:5432/pivot_facebox?sslmode=disable" \
    --auto-approve

Lint the migrations directory to run checks (this also runs in CI)

atlas migrate lint \
    --latest 100 \
    --dir "file://apps/facebox/ent/migrate/migrations" \
    --dev-url "docker://postgres/15/test?search_path=public"

After customizing a migration, regenerate the hash

atlas migrate hash \
--dir "file://internal/ent/migrate/migrations"

Atlas CLI commands (opens in a new tab).

Using Postgres, Drizzle, and Drizzle Kit

We use Drizzle as our TypeScript ORM. It comes with a CLI companion, drizzle-kit that we can use in local development, but which is not actually needed to run migrations, just to generate them.

Generate a new migration file

pnpm drizzle-kit generate:pg --config=apps/buzzbuzz/src/drizzle/drizzle.config.ts

Prior to generating a migration, it can be helpful to force your local database to match the schema files

pnpm drizzle-kit push:pg --config=apps/buzzbuzz/src/drizzle/drizzle.config.ts

Using Git Hooks

In the root of the pivot repo, you will find a folder called githooks, which contains commands that will be run whenever some actions are performed.

The purpose of using hooks is to automate some tasks that need to run whenever switching branches or pulling changes, for example (i.e. running buf generate).

To enable all git hooks, run the following command in the project root directory (only once):

git config core.hooksPath './githooks'

Emulating AWS Services with LocalStack

LocalStack runs via docker compose in the pivot repo. Be sure to allow the localstack-init container to run at first launch.

Use the LocalStack dashboard (opens in a new tab) from your browser to connect to the LocalStack docker container from a GUI. This is free of charge.

Services connect to LocalStack via the https://localhost.localstack.cloud environment veriable, not localhost directly.

Maintenance Checklist Testing