Multiplayer Feature Implementations

The following client-side features are related to Pilot/Dealer. Read those articles first.

Using Pilot/Dealer

Dealer websockets proxied through Pilot are disconnected after 60 minutes, which is longer than the expiration of a Pilot Token and therefore requires clients to obtain a new token to reconnect. Clients should refresh their Pilot Token before expiration if in use to minimize the milliseconds that the user feels the disconnect/reconnect. This token can be refreshed by simply running the Friend RPC used to get the token originally.

Key Principles

Be careful about trusting data from Pilot. We authenticate access and authorize who can be present (READ) vs who can be present and send data mutations (WRITE), but Pilot/Dealer are fundamentally just about creating a set of connected clients and pushing data around.
Pilot is for ephemeral data (other than, as discussed below, presence timestamps are persisted via NATS Jetstream publication). Never consider something persisted because you sent it through Pilot, and always trust Friend data over Pilot data. Best case scenario, Pilot is a little ahead of Friend. Worst case scenario Pilot is wrong / being abused.
Never send data you've received from other clients via Pilot to Friend. You should only ever send your own local mutations to Pilot. This is fairly trivial with YJS. It is key that you send patches that are the diff between the last know Friend state and the local state only considering your own changes.
Expect Pilot to disconnect or for the websocket to die without formally disconnecting. Ping/pong, reconnect as needed, and handle token expirations.
Keep in mind that Pilot and Dealer have no knowledge of the nuances of Facebox/Blockhead/Messenger data structures or UI. If you are editing a Database Item block, then you are present in the parent Database and you need to send your presence data to the Database blockId as well as the Database Item. The Dealer instances representing the Database and Database Item block scopes have no physical or logical overlap, nor do Pilot or Dealer understand there to be any relationship between the two. That is all client-side logic and Blockhead logic. (Dealer is actually fairly sophisticated when it comes to telling the right clients about the right things, as long as the clients properly notify Dealer about who they are and where in the application they are.)
Keep in mind that while Dealer handles block and space presence and persists it to Blockhead, Facebox also consumes those same presence messages, so even when connected to a user-scoped Dealer instance, clients should regularly push presence messages, albeit not as frequently as when sending cursor positions in space and block scopes.
Clients may receive BinaryBlockUpdates via Pilot for a blockId they aren't familiar with / aren't currently rendering. This should trigger the client to query for that blockId in Friend, as it is probably a new block that another client using Pilot/Dealer is broadcasting to the nearest ancestor that is Pilot-eligible before Friend/Blockhead have had a chance to save and notify users via Dealer of the new data. The client should query multiple times if the first time didn't return a response.

Consuming and Publishing

Clients need to connect to, publish to, consume from, and disconnect from Pilot as part of the React component rendering lifecycle. Which Pilot scope (userId, blockId or spaceId) to connect to and publish to is dependent on the specific didMount rendering context. Additionally, clients should consider that they need not have a user scope Pilot connection open at the same time as space/block, because they can receive the same server-sent updates from any scope. (See Dealer to understand what messages Dealer sends other than the messages that other clients in the same scope send to Dealer.)

Publishing

Clients publish various message types to Dealer intended for other clients in the same scope. Which Pilot scope(s) (provied via Pilot Tokens) to publish to depends on the type of message. As discussed later, multiple messages can be sent in the same websocket frame as an array of objects.

BinaryBlockUpdate patches are published to the Pilot scope for the Pilot- eligible blockId that is the nearest ancestor to the blockId being edited. For example, a Numbered List Item block rich text patch is published to the blockId for the page/goal/database item/event/etc. that it a child or descendant of, with no other Pilot-eligible blocks in between. When there is no Pilot-eligible parent, (i.e., when the patch is for a block that is rendered on the space home page) the spaceId is used as the Pilot scope to publish to. These patches are ephemeral. Clients must also publish their BinaryBlockUpdate patches to the Friend service for the blockId being edited. Publishing the nearest Pilot-eligible parent blockId is entirely unrelated to persisting changes to the server.
Presence updates (cursor position, cursor chat, lastSeenHere) are published to the Pilot scope for the Pilot-eligible blockId that is currently in full-screen or, if the user is currently interacting with a Pilot-eligible block type in inline view, to that blockId. Interacting means mutating state, such as editing a canvas block while rendering it in inline view. Additionally, presence updates are also published to the Pilot scope for the nearest Pilot-eligible ancestor of the block being rendered in full-screen or relevant block rendered in inline view. For example, a user editing a page also publishes their presence to the parent page, because the page blockId being edited will also show who is present when being rendered in link from parent view. As another example, when viewing a database item in full-screen or modal view (which is considered inline for a database item), publish presence to the parent database blockId so that clients consuming the Pilot scope for the database also can render which users are viewing which database items. When viewing the space home page, publish to the Pilot scope for the spaceId. These messages to Pilot aren't exactly ephemeral, in that Dealer persists the lastSeenAt information to NATS, and Facebox and Blockhead both consume that data.
Typing indicators and read receipts are used only in room_ scopes.

Client-side rate limiting

The Dealer service is designed for throughput of no more than 100 total RPS per scope. Therefore, each client should consider the number of clients and throttle the frequency with which it sends updates. (A Presence update and one or more block updates can be sent in the same frame)

Less than 10 other clients: send every 100ms (10 fps). (10 clients multiplied by 10 fps equals 100 requests per second)
Greater than 10 total clients, reduce send frequency so that numClients multipled by nunUpdatesPerClient is no greater than 100. (15 clients, send once every 150 ms)

Consuming

Logic for consuming is fairly simple compared to publishing.

Clients consume presence and BinaryBlockUpdate patches from the spaceId when rendering the space home page, and otherwise from the blockId for the currently full-screen block and any other Pilot-eligible blocks that are currently being rendered inline on that full-screen block.
Presence data is combined with lastSeenAt data from Friend to render the avatar stack and is otherwise used to render cursors and other ephemeral information.
BinaryBlockUpdate patches are applied to local state, but kept separate from the array of patches generated by the current user which are 1) possible to undo/redo and 2) the responsibility of this client to persist to the Friend service.
When not rendering a space or a block, clients connect to a scope for their userId. This enables clients for the same user to consume various messages from each other such as for tracking presence in audio and video calls on multiple devices.

Message Schemas

While the below schemas are shown separately, clients can and should send presence and block updates in the same websocket frame to maximize frames per second while also not exceeding 10 fps / not sending a frame less than 100ms after the last. This can be done by sending an array with one presence update and one or more binary block updates.

Presence

type presenceUpdate = {
  type: 'PRESENCE';
  // This is optional because Dealer will add it if not provided and will overwrite it if provided.
  timestamp?: number;
  // Maybe the blockId is a child of the Pilot scope blockId or maybe it is
  // simply the Pilot scope id. Either way the id is provided if the scope is
  // blockId or spaceId. If the scope is userId, an id is not provided.
  blockIdPresentIn?: string;
  // Not provided if the scope is userId.
  cursorChat?: string;
  // Not provided if the scope is userId.
  cursor?:
    | {
        characterIndex: number;
        selection?: {
          startCharacterIndex: number;
          endCharacterIndex: number;
        };
      }
    | {
        x: number;
        y: number;
        selection?: {
          start: {
            x: number;
            y: number;
          };
          end: {
            x: number;
            y: number;
          };
        };
      };
  // Not provided unless the scope is userId and the user is in a call.
  inCall?: {
    // The callId of the call that the user is in. Maybe a roomId or possibly
    // a blockId.
    callId: string;
    // The friendly name of the device that the user is in a call via.
    deviceName: string;
    // The friendly name of the call (user name or room name, for example)
    callName: string;
    // The type of call that the user is in.
    callType: 'audio' | 'video';
  };
  }
};

Binary Block Update

type binaryBlockUpdate = {
  type: 'BINARY_BLOCK_UPDATE';
  blockId: string;
  // The field being updated. Only `richTextContent`.
  field: 'richTextContent';
  patch: Uint8Array;
};

Typing Indicators and Read Receipts in Rooms

Clients can connect to Pilot with a room_ scope and send/receive typing indicators (ephemeral) and read receipts (best effort persistence from Dealer to NATS to Messenger).

type typingIndicator = {
  // Only valid if the scope is a roomId.
  type: 'TYPING_INDICATOR';
  // The userId of the user typing. Pilot ensures this matches the userId of
  // the connection.
  userId: string;
  // The roomId matching the scope. Forced to be the roomId of the scope.
  roomId: string;
  // Optional messageId representing the parent thread of the typing indicator.
  parentMessageId?: string;
  // The timestamp of when the typing indicator was sent. Set by Dealer.
  timestamp?: number;
};

type readReceipt = {
  // Only valid if the scope is a roomId.
  type: 'READ_RECIEPT';
  // The userId of the user who read. Pilot ensures this matches the userId of
  // the connection.
  userId: string;
  // The roomId matching the scope. Forced to be the roomId of the scope.
  roomId: string;
  // Message read
  messageId: string;
  // The timestamp of when the read receipt was sent. Set by Dealer.
  timestamp?: number;
};

Rich Text Editing, Text Cursors, and Selection

The presenceUpdate type above shows the format of cursors and selection in the context of rich text.

BinaryBlockUpdates are used for the richTextContent field including the field on Canvas blocks representing the block name. They are also used for richTextContent on TLD_SHAPE blocks, but not for the properties of shapes themselves. Storing canvas shapes as their own blocks does enough to avoid conflicts between multiple users editing the same canvas.

Canvas Cursors

The presenceUpdate type above shows the format of cursors and selection in the context of canvas blocks.

Avatar Stacks

Clients display an avatar stack by merging a few different data sources:

The presence array retrieved from the block type in Friend is used as the base.
The presence array obtained through Dealer is merged on top of this data, meaning that the data in Dealer is considered fresher than Friend to the degree an individual lastSeenAt value is newer.

Cursor Chat

When a client sends a presence message that includes a cursor chat string, it is rendered until 1) the client sends another presence message that includes a cursor chat string (including an empty string, which clears the cursor chat value) or 2) 10 seconds go by.

Undo/Redo

Undo/Redo in Pivot has two experiences, of which only the second has any relation to Dealer:

If a user has their cursor in a message input (for a comment on a block, a post in a post room, or a chat in a chat room), undo/redo applies to that text input specifically.
When a user is editing blocks in any context, their edits are kept in a separate stack of operations that includes both BinaryBlockUpdate patches as well as atomic updates to block properties (Friend mutations applied optimistically, which may or may not be acknowledged by the server at any give time). Because we store the opposite operation of each atomic update in this array, we can undo an automatic operation simply by doing the opposite, and adding the opposite of that (the original) to the undo stack. The undo stack is append only.

When a user is otherwise just navigating the application and taking action by submitting traditional forms/clicking traditional action buttons (for example changing their user settings or sending a message), there is no undo/redo experience. That is to say, you cannot use Cmd + Z to undo a change you made to your preferredEmail, other than the fact that you can use undo in text inputs. Likewise, you cannot click 'Send' on a message and then use Cmd + Z to un-send the message -- this is a confusing use of undo/redo because the UI includes specific delete and edit concepts to operate on an already sent message. As a final example, you can't undo adding a block to your sidebar (favorites) because that's what the 'unfavorite' action is for.

Undo/redo is different from navigating backwards and forwards across routes. That is totally unrelated to undo/redo. Navigating to a different route clears the undo/redo stack, because as noted above, the undo/redo stack is always tied to something that is being currently rendered. Cmd + Z should never 1) undo something that isn't being rendered or 2) trigger a route change. (Though in theory a cleared stack can be saved in state, so upon returning to the same route moments later, the prior stack could still be present for that route.)

Marketing Site Pivdraw