# gRPC

Course: [API](https://www.devtutorials.ai/learn/building-apis) · Section: Beyond REST
Source: https://www.devtutorials.ai/learn/building-apis/beyond-rest/grpc

> Let services talk fast with small binary protobuf messages and a strict .proto contract.

**gRPC** is an API style designed for communication between services.

Instead of sending JSON text, gRPC sends compact **binary messages** encoded using **Protocol Buffers (protobuf)**. Both the client and server agree on the structure of every message through a shared contract file called a **`.proto`** file. Because the message format is predefined, gRPC can send data more efficiently than JSON and is commonly used for communication between internal services.

## Watch an Internal Call Carry Its Own Labels

As Brew & Bytes grows, the Order Service no longer processes payments itself. Instead, it calls a separate Payment Service. The services currently communicate using JSON. Press **Run**.

Notice the response size:

```http
X-Body-Bytes: 87
```

Much of the message consists of field names:

```json
{
  "paymentId": "...",
  "currency": "INR",
  "status": "approved"
}
```

The Payment Service already knows these field names. The Order Service already knows these field names. Transmitting the same labels repeatedly is unnecessary work.

**In simple words:** JSON repeatedly sends field names. gRPC sends compact binary data because both sides already know the structure.

## TL;DR

- gRPC uses **Protocol Buffers (protobuf)**.
- Messages are encoded as compact binary bytes.
- The **`.proto` file** defines the contract.
- Field names are never sent across the network.
- Every field has a unique field number.
- Small numbers use very little space through **varint encoding**.
- Never change or reuse field numbers.
- Removed field numbers should be marked as `reserved`.
- gRPC runs on **HTTP/2**.
- gRPC supports:
  - Unary calls
  - Streaming
- gRPC is best suited for internal service-to-service communication.
- Browsers typically require **gRPC-Web** rather than raw gRPC.

## A Real-World Example

Imagine restaurant staff communicating inside a busy kitchen. A waiter could write:

```text
Dear kitchen,

Please prepare two lattes for table four.

Thank you.
```

This contains a lot of unnecessary text. Instead, kitchens often use shorthand:

```text
T4 2xLAT
```

Everyone already understands the notation. It is faster to write and faster to read. The `.proto` file is the shared codebook. The protobuf message is the shorthand note.

## What You Will Learn

In this lesson, you will learn:

1. Why gRPC is efficient for internal communication.
2. How `.proto` contracts work.
3. How protobuf encodes data into bytes.
4. How protobuf decodes bytes back into values.
5. How to safely evolve a contract.
6. How unary and streaming calls differ.

**What runs here, and what does not.** gRPC needs HTTP/2 and generated code, and neither can run in this browser. So the cells below do the part that *can* run: they build and read **real protobuf bytes by hand**. The bytes are exactly what the official protobuf library produces for the same message. The HTTP/2 transport and the generated client and server classes are shown as **read-only** code. No cell here makes a real gRPC call.

## The Contract: A `.proto` File

Every gRPC application starts with a `.proto` file.

Example:

```protobuf
syntax = "proto3";

package brew.payments;

message PaymentRequest {
  int32 order_id = 1;
  int32 amount = 2;
  string currency = 3;
}

message PaymentReply {
  string payment_id = 1;
  string status = 2;
}

message StatusRequest {
  int32 order_id = 1;
}

message StatusUpdate {
  int32 order_id = 1;
  string status = 2;
}

service PaymentService {
  rpc Charge (PaymentRequest)
      returns (PaymentReply);

  rpc GetStatus (StatusRequest)
      returns (StatusUpdate);

  rpc WatchStatus (StatusRequest)
      returns (stream StatusUpdate);
}
```

### Field Numbers

Consider:

```protobuf
int32 order_id = 1;
```

The value:

```protobuf
= 1
```

is the field number.

It is not a default value.

Field numbers identify fields on the wire.

Example:

```protobuf
order_id = 1
amount   = 2
currency = 3
```

Only these numbers are transmitted.

The field names never travel across the network.

### Service Definitions

A service acts much like a collection of endpoints.

Example:

```protobuf
service PaymentService {
  rpc Charge (...)
}
```

The `Charge` RPC behaves similarly to a REST endpoint but uses protobuf messages instead of JSON.

## Encode a Message into Bytes

Suppose we create:

```json
{
  "orderId": 101,
  "amount": 440,
  "currency": "INR"
}
```

In protobuf, each field is stored as: Tag + Value.

The tag contains:

```text
Field Number
+
Wire Type
```

Formula:

```text
(field_number << 3) | wire_type
```

### order_id = 101

Tag: 08

Value: 65

### amount = 440

Tag: 10

Value: B8 03

### currency = "INR"

Tag: 1A

Length: 03

Bytes: 49 4E 52

Total protobuf size: 10 bytes

Equivalent JSON: 45 bytes

The savings become significant when millions of messages are exchanged.

## Decode Bytes Back into Data

Decoding performs the reverse process.

The reader:

1. Reads a tag.
2. Extracts the field number.
3. Determines the wire type.
4. Reads the value.

Result:

```text
Field 1 → 101
Field 2 → 440
Field 3 → INR
```

Notice that the bytes never store:

```text
order_id
amount
currency
```

The `.proto` file provides those meanings. Without the contract, the data cannot be interpreted correctly.

In proto3, fields with default values are often omitted from the message.

For example:

```protobuf
int32 points = 0;
```

may not be transmitted at all.

If your application must distinguish between 0 and Not provided

use:

```protobuf
optional
```

fields.

## Evolving the Contract Safely

Field numbers are permanent. Changing their meaning can produce silent data corruption. Consider this change:

Original:

```protobuf
int32 amount = 2;
```

Updated:

```protobuf
int32 discount = 2;
int32 amount = 4;
```

Old services still interpret field:

```protobuf
2
```

as:

```text
amount
```

Result: amount = 40

instead of: amount = 440

No exception occurs. No validation error appears. The payment service silently processes the wrong value.

Never change or reuse a field number. Field numbers are part of the public contract.

### Safe Removal

When removing a field:

```protobuf
message PaymentRequest {
  reserved 2;
  reserved "amount_old";

  int32 order_id = 1;
  string currency = 3;
  int32 amount = 4;
  int32 discount = 5;
}
```

Now field number `2` can never be reused accidentally.

### Safe Changes

Safe contract changes include:

- Adding a new field with a new number
- Reserving removed fields
- Adding new messages
- Adding new service methods

Unsafe changes include:

- Reusing field numbers
- Changing field meaning
- Renumbering fields

## Unary and Streaming Calls

gRPC supports multiple communication patterns.

### Unary Calls

One request. One response.

Example:

- Client → Charge
- Server → PaymentReply

This works much like a traditional REST request.

### Server Streaming

One request. Many responses.

Example:

```text
Client → WatchStatus

Server → received
Server → preparing
Server → ready
```

The connection remains open while updates arrive.

Streaming avoids repeated polling and supports real-time updates efficiently.

### gRPC Framing

Each protobuf message is placed inside a gRPC frame.

Frame structure:

```text
1 byte  Compression Flag
4 bytes Message Length
N bytes Message Data
```

Example:

```text
00 00 00 00 0D
```

means:

```text
Not compressed
13-byte message follows
```

Streaming simply sends multiple frames on the same HTTP/2 connection.

## Real gRPC in ASP.NET Core

A real server typically looks like:

```csharp
builder.Services.AddGrpc();

var app = builder.Build();

app.MapGrpcService<PaymentService>();

app.Run();
```

Generated classes come directly from the `.proto` contract. The implementation focuses only on business logic.

```csharp
public override Task<PaymentReply> Charge(
    PaymentRequest request,
    ServerCallContext context)
{
    return Task.FromResult(
        new PaymentReply
        {
            PaymentId = $"pay-{request.OrderId}",
            Status = "approved"
        });
}
```

The framework handles:

- Serialization
- Deserialization
- HTTP/2 transport
- Generated client code

## REST vs gRPC

| Feature | REST | gRPC |
|---|---|---|
| Format | JSON | Protocol Buffers |
| Readability | Human-readable | Binary |
| Message Size | Larger | Smaller |
| Browser Support | Excellent | Requires gRPC-Web |
| Public APIs | Excellent | Rare |
| Internal Services | Good | Excellent |
| Streaming | Limited | Built-in |
| Contract | Optional | Required (`.proto`) |

### When to Use gRPC

gRPC is a strong choice when:

- Both client and server are owned by your organization.
- Performance matters.
- Services communicate frequently.
- Strong contracts are important.
- Streaming is required.

### When REST Is Better

REST is usually a better choice when:

- External developers consume the API.
- Requests are inspected frequently.
- Browser compatibility is important.
- Simplicity is preferred.

## Where You See This

- Large microservice architectures use gRPC between services.
- Google Cloud services heavily rely on gRPC.
- Kubernetes components communicate using gRPC.
- Many .NET teams use `Grpc.AspNetCore`.
- JavaScript and TypeScript teams often use `@grpc/grpc-js` or Connect.

## Common mistakes

| Symptom | Cause | Fix |
|---|---|---|
| Services interpret values incorrectly after deployment | A field number was changed or reused | Never reuse field numbers; reserve removed ones |
| Browser requests fail | Browsers cannot use raw gRPC directly | Use gRPC-Web or a REST gateway |
| A real value of `0` is treated as missing | Default proto3 behavior | Use `optional` fields |
| Services disagree about message formats | Different `.proto` versions are deployed | Share one contract and regenerate clients |
| Messages are larger than expected | JSON is used for high-volume internal traffic | Switch internal service calls to gRPC |
| Status updates require repeated polling | Unary calls are used for live updates | Use gRPC streaming |

## Exercises

## What to Remember

- gRPC sends small binary **protobuf** messages. The **`.proto`** file is the shared contract.
- Each field is a **tag** plus a value. `tag = (field_number << 3) | wire_type`. Names never travel.
- Numbers are **varints**: 7 bits in each byte, and the top bit means "more bytes follow".
- **Never reuse a field number.** Add new numbers, and `reserve` the removed ones. Old readers skip unknown fields safely.
- gRPC supports **unary** and **streaming** calls over HTTP/2. Each message gets a 5-byte frame header.
- Use REST for public and browser callers, and use gRPC between your own services.

**Next:** REST vs GraphQL vs gRPC. You will put the three styles side by side, and you will learn how to pick the right one for each job.

using System.Text.Json;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

// An internal endpoint: the order service asks the payment service to charge an order.
// Only our own services call it, many times a second. It speaks JSON, like a public API.
app.MapPost("/payments", (PaymentRequest req, HttpContext ctx) =>
{
    var reply = new { paymentId = "pay-" + req.OrderId, orderId = req.OrderId, amount = req.Amount, currency = req.Currency, status = "approved" };

    // Count the bytes of the JSON text we send back.
    // Every field name ("paymentId", "currency"...) travels in every single message.
    string json = JsonSerializer.Serialize(reply);
    ctx.Response.Headers["X-Body-Bytes"] = Encoding.UTF8.GetByteCount(json).ToString();
    return Results.Ok(reply);
});

app.Run();

record PaymentRequest(int OrderId, int Amount, string Currency);`;

const app = new Hono();

// An internal endpoint: the order service asks the payment service to charge an order.
// Only our own services call it, many times a second. It speaks JSON, like a public API.
app.post("/payments", async (c) => {
  const req = await c.req.json();
  const reply = { paymentId: "pay-" + req.orderId, orderId: req.orderId, amount: req.amount, currency: req.currency, status: "approved" };

  // Count the bytes of the JSON text we send back.
  // Every field name ("paymentId", "currency"...) travels in every single message.
  const json = JSON.stringify(reply);
  c.header("X-Body-Bytes", String(new TextEncoder().encode(json).length));
  return c.json(reply, 200);
});

Content-Type: application/json

{"orderId":101,"amount":440,"currency":"INR"}

###
POST /payments
Content-Type: application/json

{"orderId":102,"amount":120,"currency":"INR"}`;

// Protobuf writes each field as: a tag, then the value.
// tag = (field_number << 3) | wire_type. Wire type 0 = varint (number), 2 = length + bytes.
static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; } // 7 bits per byte, top bit = "more"
    o.Add((byte)v);
    return o;
}
static List<byte> IntField(int field, ulong v) => [.. Varint((ulong)(field << 3 | 0)), .. Varint(v)];
static List<byte> StringField(int field, string s)
{
    byte[] b = Encoding.UTF8.GetBytes(s);
    return [.. Varint((ulong)(field << 3 | 2)), .. Varint((ulong)b.Length), .. b];
}
static string Hex(IEnumerable<byte> b) => string.Join(" ", b.Select(x => x.ToString("X2")));

// message PaymentRequest { int32 order_id = 1; int32 amount = 2; string currency = 3; }
var f1 = IntField(1, 101);
var f2 = IntField(2, 440);
var f3 = StringField(3, "INR");
Console.WriteLine("field 1 order_id = 101   -> " + Hex(f1));
Console.WriteLine("field 2 amount   = 440   -> " + Hex(f2));
Console.WriteLine("field 3 currency = \\"INR\\" -> " + Hex(f3));

List<byte> bytes = [.. f1, .. f2, .. f3];
Console.WriteLine("protobuf: " + Hex(bytes));
Console.WriteLine("protobuf size: " + bytes.Count + " bytes");

string json = "{\\"orderId\\":101,\\"amount\\":440,\\"currency\\":\\"INR\\"}";
Console.WriteLine("JSON:     " + json);
Console.WriteLine("JSON size: " + Encoding.UTF8.GetByteCount(json) + " bytes");`;

// tag = (field_number << 3) | wire_type. Wire type 0 = varint (number), 2 = length + bytes.
function varint(v: number): number[] {
  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); } // 7 bits per byte, top bit = "more"
  o.push(v);
  return o;
}
const intField = (field: number, v: number) => [...varint((field << 3) | 0), ...varint(v)];
function stringField(field: number, s: string): number[] {
  const b = [...new TextEncoder().encode(s)];
  return [...varint((field << 3) | 2), ...varint(b.length), ...b];
}
const hex = (b: number[]) => b.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" ");

// message PaymentRequest { int32 order_id = 1; int32 amount = 2; string currency = 3; }
const f1 = intField(1, 101);
const f2 = intField(2, 440);
const f3 = stringField(3, "INR");
console.log("field 1 order_id = 101   -> " + hex(f1));
console.log("field 2 amount   = 440   -> " + hex(f2));
console.log("field 3 currency = \\"INR\\" -> " + hex(f3));

const bytes = [...f1, ...f2, ...f3];
console.log("protobuf: " + hex(bytes));
console.log("protobuf size: " + bytes.length + " bytes");

const json = "{\\"orderId\\":101,\\"amount\\":440,\\"currency\\":\\"INR\\"}";
console.log("JSON:     " + json);
console.log("JSON size: " + new TextEncoder().encode(json).length + " bytes");`;

// The 10 bytes from the last cell: a PaymentRequest.
byte[] bytes = Convert.FromHexString("0865 10B803 1A03494E52".Replace(" ", ""));
int pos = 0;

// Read one varint: take 7 bits from each byte until a byte has its top bit off.
ulong ReadVarint()
{
    ulong v = 0; int shift = 0;
    while (true)
    {
        byte b = bytes[pos++];
        v |= (ulong)(b & 0x7F) << shift;
        if ((b & 0x80) == 0) return v;
        shift += 7;
    }
}

while (pos < bytes.Length)
{
    ulong tag = ReadVarint();
    int field = (int)(tag >> 3);   // the high bits are the field number
    int wire = (int)(tag & 7);     // the low 3 bits say how to read the value
    if (wire == 0)
        Console.WriteLine($"tag {tag:X2} -> field {field}, wire 0 (varint), value {ReadVarint()}");
    else if (wire == 2)
    {
        int len = (int)ReadVarint();
        string text = Encoding.UTF8.GetString(bytes, pos, len);
        pos += len;
        Console.WriteLine($"tag {tag:X2} -> field {field}, wire 2 (length {len}), value \\"{text}\\"");
    }
}
Console.WriteLine("No names in the bytes. The .proto file says field 1 = order_id, 2 = amount, 3 = currency.");`;

const bytes = "0865 10B803 1A03494E52".replace(/ /g, "").match(/../g)!.map((h) => parseInt(h, 16));
let pos = 0;

// Read one varint: take 7 bits from each byte until a byte has its top bit off.
function readVarint(): number {
  let v = 0, mul = 1;
  while (true) {
    const b = bytes[pos++];
    v += (b & 0x7f) * mul;
    if ((b & 0x80) === 0) return v;
    mul *= 128;
  }
}
const h2 = (n: number) => n.toString(16).toUpperCase().padStart(2, "0");

while (pos < bytes.length) {
  const tag = readVarint();
  const field = tag >> 3; // the high bits are the field number
  const wire = tag & 7;   // the low 3 bits say how to read the value
  if (wire === 0) {
    console.log(\`tag \${h2(tag)} -> field \${field}, wire 0 (varint), value \${readVarint()}\`);
  } else if (wire === 2) {
    const len = readVarint();
    const text = new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len)));
    pos += len;
    console.log(\`tag \${h2(tag)} -> field \${field}, wire 2 (length \${len}), value "\${text}"\`);
  }
}
console.log("No names in the bytes. The .proto file says field 1 = order_id, 2 = amount, 3 = currency.");`;

static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; }
    o.Add((byte)v);
    return o;
}
static List<byte> IntField(int field, ulong v) => [.. Varint((ulong)(field << 3 | 0)), .. Varint(v)];
static List<byte> StringField(int field, string s)
{
    byte[] b = Encoding.UTF8.GetBytes(s);
    return [.. Varint((ulong)(field << 3 | 2)), .. Varint((ulong)b.Length), .. b];
}

// NEW writer (v2 of the .proto). Someone "tidied up": amount moved to field 4,
// and field 2 was reused for a new discount field.
List<byte> bytes = [.. IntField(1, 101), .. IntField(2, 40), .. StringField(3, "INR"), .. IntField(4, 440)];
Console.WriteLine("v2 writer sends: order_id 101, discount 40, currency INR, amount 440");

// OLD reader (v1 of the .proto) still thinks field 2 is amount.
var oldNames = new Dictionary<int, string> { [1] = "order_id", [2] = "amount", [3] = "currency" };
int pos = 0;
ulong ReadVarint() { ulong v = 0; int s = 0; while (true) { byte b = bytes[pos++]; v |= (ulong)(b & 0x7F) << s; if ((b & 0x80) == 0) return v; s += 7; } }

while (pos < bytes.Count)
{
    ulong tag = ReadVarint();
    int field = (int)(tag >> 3), wire = (int)(tag & 7);
    string value;
    if (wire == 0) value = ReadVarint().ToString();
    else { int len = (int)ReadVarint(); value = "\\"" + Encoding.UTF8.GetString(bytes.GetRange(pos, len).ToArray()) + "\\""; pos += len; }
    if (oldNames.TryGetValue(field, out var name)) Console.WriteLine($"v1 reader: {name} = {value}");
    else Console.WriteLine($"v1 reader: field {field} is unknown, skipped");
}
Console.WriteLine("Result: the old service charges 40 instead of 440. No error, just wrong money.");`;

  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); }
  o.push(v);
  return o;
}
const intField = (field: number, v: number) => [...varint((field << 3) | 0), ...varint(v)];
function stringField(field: number, s: string): number[] {
  const b = [...new TextEncoder().encode(s)];
  return [...varint((field << 3) | 2), ...varint(b.length), ...b];
}

// NEW writer (v2 of the .proto). Someone "tidied up": amount moved to field 4,
// and field 2 was reused for a new discount field.
const bytes = [...intField(1, 101), ...intField(2, 40), ...stringField(3, "INR"), ...intField(4, 440)];
console.log("v2 writer sends: order_id 101, discount 40, currency INR, amount 440");

// OLD reader (v1 of the .proto) still thinks field 2 is amount.
const oldNames: Record<number, string> = { 1: "order_id", 2: "amount", 3: "currency" };
let pos = 0;
function readVarint(): number { let v = 0, m = 1; while (true) { const b = bytes[pos++]; v += (b & 0x7f) * m; if ((b & 0x80) === 0) return v; m *= 128; } }

while (pos < bytes.length) {
  const tag = readVarint();
  const field = tag >> 3, wire = tag & 7;
  let value: string;
  if (wire === 0) value = String(readVarint());
  else { const len = readVarint(); value = "\\"" + new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len))) + "\\""; pos += len; }
  if (field in oldNames) console.log(\`v1 reader: \${oldNames[field]} = \${value}\`);
  else console.log(\`v1 reader: field \${field} is unknown, skipped\`);
}
console.log("Result: the old service charges 40 instead of 440. No error, just wrong money.");`;

static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; }
    o.Add((byte)v);
    return o;
}
static string Hex(IEnumerable<byte> b) => string.Join(" ", b.Select(x => x.ToString("X2")));

// message StatusUpdate { int32 order_id = 1; string status = 2; }
static List<byte> StatusUpdate(int orderId, string status)
{
    byte[] s = Encoding.UTF8.GetBytes(status);
    return [.. Varint(1 << 3 | 0), .. Varint((ulong)orderId), .. Varint(2 << 3 | 2), .. Varint((ulong)s.Length), .. s];
}

// gRPC puts a 5-byte header before every message:
// 1 byte "compressed?" flag (0 = no), then the message length as 4 bytes, big-endian.
static string Frame(List<byte> msg)
{
    int n = msg.Count;
    byte[] header = [0, (byte)(n >> 24), (byte)(n >> 16), (byte)(n >> 8), (byte)n];
    return Hex(header) + " | " + Hex(msg);
}

Console.WriteLine("[simulated transport: frames are built here, nothing is sent over HTTP/2]");
Console.WriteLine("Unary GetStatus(101): one request, one reply");
Console.WriteLine("  reply:    " + Frame(StatusUpdate(101, "preparing")));

// Server streaming: one request, then the server keeps sending frames on the same call.
Console.WriteLine("Server streaming WatchStatus(101): one request, many replies");
IEnumerable<List<byte>> WatchStatus(int orderId)
{
    foreach (var s in new[] { "received", "preparing", "ready" }) yield return StatusUpdate(orderId, s);
}
int i = 1;
foreach (var msg in WatchStatus(101)) Console.WriteLine($"  frame {i++}:  " + Frame(msg));
Console.WriteLine("  (end of stream: the server closes the call with status OK)");`;

  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); }
  o.push(v);
  return o;
}
const hex = (b: number[]) => b.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" ");

// message StatusUpdate { int32 order_id = 1; string status = 2; }
function statusUpdate(orderId: number, status: string): number[] {
  const s = [...new TextEncoder().encode(status)];
  return [...varint((1 << 3) | 0), ...varint(orderId), ...varint((2 << 3) | 2), ...varint(s.length), ...s];
}

// gRPC puts a 5-byte header before every message:
// 1 byte "compressed?" flag (0 = no), then the message length as 4 bytes, big-endian.
function frame(msg: number[]): string {
  const n = msg.length;
  const header = [0, (n >>> 24) & 255, (n >>> 16) & 255, (n >>> 8) & 255, n & 255];
  return hex(header) + " | " + hex(msg);
}

console.log("[simulated transport: frames are built here, nothing is sent over HTTP/2]");
console.log("Unary GetStatus(101): one request, one reply");
console.log("  reply:    " + frame(statusUpdate(101, "preparing")));

// Server streaming: one request, then the server keeps sending frames on the same call.
console.log("Server streaming WatchStatus(101): one request, many replies");
function* watchStatus(orderId: number) {
  for (const s of ["received", "preparing", "ready"]) yield statusUpdate(orderId, s);
}
let i = 1;
for (const msg of watchStatus(101)) console.log(\`  frame \${i++}:  \` + frame(msg));
console.log("  (end of stream: the server closes the call with status OK)");`;

  {
    prompt: "Varint() only works for numbers below 128. Fix it so that 150 prints 96 01 and 300 prints AC 02.",
    hint: "Loop while v is 0x80 or more. In each turn, add the low 7 bits with the top bit turned on, and then shift v right by 7. After the loop, add the last byte.",
    explain: "150 is 1 0010110 in binary. The low 7 bits (0010110) go first, with the top bit turned on: 0x96. The rest (1) comes next: 0x01. Small numbers stay at one byte, which is why protobuf is so compact.",
    csharp: { starter: `// A varint stores a number in 7-bit groups, lowest group first.
// Every byte except the last has its top bit (0x80) turned on, meaning "more bytes follow".
static List<byte> Varint(ulong v)
{
    // TODO: this only works for numbers below 128.
    // Loop: while v >= 0x80, add (v & 0x7F) | 0x80, then shift v right by 7. Then add the last byte.
    return [(byte)v];
}
static string Hex(IEnumerable<byte> b) => string.Join(" ", b.Select(x => x.ToString("X2")));

// Expected: 1 -> 01, 150 -> 96 01, 300 -> AC 02
foreach (ulong n in new ulong[] { 1, 150, 300 }) Console.WriteLine($"{n} -> {Hex(Varint(n))}");`, solution: `// A varint stores a number in 7-bit groups, lowest group first.
// Every byte except the last has its top bit (0x80) turned on, meaning "more bytes follow".
static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; }
    o.Add((byte)v);
    return o;
}
static string Hex(IEnumerable<byte> b) => string.Join(" ", b.Select(x => x.ToString("X2")));

// Expected: 1 -> 01, 150 -> 96 01, 300 -> AC 02
foreach (ulong n in new ulong[] { 1, 150, 300 }) Console.WriteLine($"{n} -> {Hex(Varint(n))}");`, mode: "console" },
    react: { starter: `// A varint stores a number in 7-bit groups, lowest group first.
// Every byte except the last has its top bit (0x80) turned on, meaning "more bytes follow".
function varint(v: number): number[] {
  // TODO: this only works for numbers below 128.
  // Loop: while v >= 0x80, push (v & 0x7f) | 0x80, then divide v by 128 (drop the low 7 bits). Then push the last byte.
  return [v & 0xff];
}
const hex = (b: number[]) => b.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" ");

// Expected: 1 -> 01, 150 -> 96 01, 300 -> AC 02
for (const n of [1, 150, 300]) console.log(\`\${n} -> \${hex(varint(n))}\`);`, solution: `// A varint stores a number in 7-bit groups, lowest group first.
// Every byte except the last has its top bit (0x80) turned on, meaning "more bytes follow".
function varint(v: number): number[] {
  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); }
  o.push(v);
  return o;
}
const hex = (b: number[]) => b.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" ");

// Expected: 1 -> 01, 150 -> 96 01, 300 -> AC 02
for (const n of [1, 150, 300]) console.log(\`\${n} -> \${hex(varint(n))}\`);`, mode: "console" }
  },
  {
    prompt: "Version 2 of the .proto adds optional int32 tip = 4. Write the tip (50) as field 4, and check that the old v1 reader still works.",
    hint: "Add IntField(4, 50) (intField in TypeScript) to the bytes. The tag is (4 << 3) | 0 = 0x20.",
    explain: "The new bytes end with 20 32. The v1 reader does not know field 4, but the wire type tells it how long the value is, so it skips the field and reads the rest correctly. Adding new numbers is always safe.",
    csharp: { starter: `using System.Text;

static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; }
    o.Add((byte)v);
    return o;
}
static List<byte> IntField(int field, ulong v) => [.. Varint((ulong)(field << 3 | 0)), .. Varint(v)];
static List<byte> StringField(int field, string s)
{
    byte[] b = Encoding.UTF8.GetBytes(s);
    return [.. Varint((ulong)(field << 3 | 2)), .. Varint((ulong)b.Length), .. b];
}

// The v2 writer: the new checkout screen lets Asha add a tip.
List<byte> bytes = [.. IntField(1, 101), .. IntField(2, 440), .. StringField(3, "INR")];
// TODO: v2 of the .proto adds "optional int32 tip = 4;". Append the tip (50) as field 4.
Console.WriteLine("v2 bytes: " + string.Join(" ", bytes.Select(x => x.ToString("X2"))));

// The v1 reader was built before tips existed. It must not crash on field 4.
var known = new Dictionary<int, string> { [1] = "order_id", [2] = "amount", [3] = "currency" };
int pos = 0;
ulong ReadVarint() { ulong v = 0; int s = 0; while (true) { byte b = bytes[pos++]; v |= (ulong)(b & 0x7F) << s; if ((b & 0x80) == 0) return v; s += 7; } }
while (pos < bytes.Count)
{
    ulong tag = ReadVarint();
    int field = (int)(tag >> 3), wire = (int)(tag & 7);
    string value;
    if (wire == 0) value = ReadVarint().ToString();
    else { int len = (int)ReadVarint(); value = Encoding.UTF8.GetString(bytes.GetRange(pos, len).ToArray()); pos += len; }
    // The wire type tells the reader how long the value is, so it can step over a field it does not know.
    Console.WriteLine(known.TryGetValue(field, out var name) ? $"v1 reader: {name} = {value}" : $"v1 reader: skipped unknown field {field} (wire {wire})");
}`, solution: `using System.Text;

static List<byte> Varint(ulong v)
{
    var o = new List<byte>();
    while (v >= 0x80) { o.Add((byte)((v & 0x7F) | 0x80)); v >>= 7; }
    o.Add((byte)v);
    return o;
}
static List<byte> IntField(int field, ulong v) => [.. Varint((ulong)(field << 3 | 0)), .. Varint(v)];
static List<byte> StringField(int field, string s)
{
    byte[] b = Encoding.UTF8.GetBytes(s);
    return [.. Varint((ulong)(field << 3 | 2)), .. Varint((ulong)b.Length), .. b];
}

// The v2 writer: the new checkout screen lets Asha add a tip.
// v2 of the .proto adds "optional int32 tip = 4;". A new number, so nothing old changes meaning.
List<byte> bytes = [.. IntField(1, 101), .. IntField(2, 440), .. StringField(3, "INR"), .. IntField(4, 50)];
Console.WriteLine("v2 bytes: " + string.Join(" ", bytes.Select(x => x.ToString("X2"))));

// The v1 reader was built before tips existed. It must not crash on field 4.
var known = new Dictionary<int, string> { [1] = "order_id", [2] = "amount", [3] = "currency" };
int pos = 0;
ulong ReadVarint() { ulong v = 0; int s = 0; while (true) { byte b = bytes[pos++]; v |= (ulong)(b & 0x7F) << s; if ((b & 0x80) == 0) return v; s += 7; } }
while (pos < bytes.Count)
{
    ulong tag = ReadVarint();
    int field = (int)(tag >> 3), wire = (int)(tag & 7);
    string value;
    if (wire == 0) value = ReadVarint().ToString();
    else { int len = (int)ReadVarint(); value = Encoding.UTF8.GetString(bytes.GetRange(pos, len).ToArray()); pos += len; }
    // The wire type tells the reader how long the value is, so it can step over a field it does not know.
    Console.WriteLine(known.TryGetValue(field, out var name) ? $"v1 reader: {name} = {value}" : $"v1 reader: skipped unknown field {field} (wire {wire})");
}`, mode: "console" },
    react: { starter: `function varint(v: number): number[] {
  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); }
  o.push(v);
  return o;
}
const intField = (field: number, v: number) => [...varint((field << 3) | 0), ...varint(v)];
function stringField(field: number, s: string): number[] {
  const b = [...new TextEncoder().encode(s)];
  return [...varint((field << 3) | 2), ...varint(b.length), ...b];
}

// The v2 writer: the new checkout screen lets Asha add a tip.
const bytes = [...intField(1, 101), ...intField(2, 440), ...stringField(3, "INR")];
// TODO: v2 of the .proto adds "optional int32 tip = 4;". Append the tip (50) as field 4.
console.log("v2 bytes: " + bytes.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" "));

// The v1 reader was built before tips existed. It must not crash on field 4.
const known: Record<number, string> = { 1: "order_id", 2: "amount", 3: "currency" };
let pos = 0;
function readVarint(): number { let v = 0, m = 1; while (true) { const b = bytes[pos++]; v += (b & 0x7f) * m; if ((b & 0x80) === 0) return v; m *= 128; } }
while (pos < bytes.length) {
  const tag = readVarint();
  const field = tag >> 3, wire = tag & 7;
  let value: string;
  if (wire === 0) value = String(readVarint());
  else { const len = readVarint(); value = new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len))); pos += len; }
  // The wire type tells the reader how long the value is, so it can step over a field it does not know.
  console.log(field in known ? \`v1 reader: \${known[field]} = \${value}\` : \`v1 reader: skipped unknown field \${field} (wire \${wire})\`);
}`, solution: `function varint(v: number): number[] {
  const o: number[] = [];
  while (v >= 0x80) { o.push((v & 0x7f) | 0x80); v = Math.floor(v / 128); }
  o.push(v);
  return o;
}
const intField = (field: number, v: number) => [...varint((field << 3) | 0), ...varint(v)];
function stringField(field: number, s: string): number[] {
  const b = [...new TextEncoder().encode(s)];
  return [...varint((field << 3) | 2), ...varint(b.length), ...b];
}

// The v2 writer: the new checkout screen lets Asha add a tip.
// v2 of the .proto adds "optional int32 tip = 4;". A new number, so nothing old changes meaning.
const bytes = [...intField(1, 101), ...intField(2, 440), ...stringField(3, "INR"), ...intField(4, 50)];
console.log("v2 bytes: " + bytes.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" "));

// The v1 reader was built before tips existed. It must not crash on field 4.
const known: Record<number, string> = { 1: "order_id", 2: "amount", 3: "currency" };
let pos = 0;
function readVarint(): number { let v = 0, m = 1; while (true) { const b = bytes[pos++]; v += (b & 0x7f) * m; if ((b & 0x80) === 0) return v; m *= 128; } }
while (pos < bytes.length) {
  const tag = readVarint();
  const field = tag >> 3, wire = tag & 7;
  let value: string;
  if (wire === 0) value = String(readVarint());
  else { const len = readVarint(); value = new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len))); pos += len; }
  // The wire type tells the reader how long the value is, so it can step over a field it does not know.
  console.log(field in known ? \`v1 reader: \${known[field]} = \${value}\` : \`v1 reader: skipped unknown field \${field} (wire \${wire})\`);
}`, mode: "console" }
  },
  {
    prompt: "The decoder prints fields 8, 16 and 26. Fix it so that it prints the real field numbers 1, 2 and 3.",
    hint: "The tag mixes two things together. Shift it right by 3 to drop the wire type.",
    explain: "A tag is (field_number << 3) | wire_type. 0x08 >> 3 = 1, 0x10 >> 3 = 2, and 0x1A >> 3 = 3. The low 3 bits (0, 0 and 2) are the wire types.",
    csharp: { starter: `using System.Text;

// A PaymentRequest arrived from the order service. Decode it by hand.
byte[] bytes = Convert.FromHexString("08 66 10 78 1A 03 49 4E 52".Replace(" ", ""));
Console.WriteLine("bytes: " + string.Join(" ", bytes.Select(x => x.ToString("X2"))));
int pos = 0;
ulong ReadVarint() { ulong v = 0; int s = 0; while (true) { byte b = bytes[pos++]; v |= (ulong)(b & 0x7F) << s; if ((b & 0x80) == 0) return v; s += 7; } }

// Expected: field 1 = 102, field 2 = 120, field 3 = "INR"
while (pos < bytes.Length)
{
    ulong tag = ReadVarint();
    // TODO: this prints the whole tag as the field number. The field number is the tag shifted right by 3.
    int field = (int)tag;
    int wire = (int)(tag & 7);
    if (wire == 0) Console.WriteLine($"field {field} (varint) = {ReadVarint()}");
    else { int len = (int)ReadVarint(); Console.WriteLine($"field {field} (length {len}) = \\"{Encoding.UTF8.GetString(bytes, pos, len)}\\""); pos += len; }
}`, solution: `using System.Text;

// A PaymentRequest arrived from the order service. Decode it by hand.
byte[] bytes = Convert.FromHexString("08 66 10 78 1A 03 49 4E 52".Replace(" ", ""));
Console.WriteLine("bytes: " + string.Join(" ", bytes.Select(x => x.ToString("X2"))));
int pos = 0;
ulong ReadVarint() { ulong v = 0; int s = 0; while (true) { byte b = bytes[pos++]; v |= (ulong)(b & 0x7F) << s; if ((b & 0x80) == 0) return v; s += 7; } }

// Expected: field 1 = 102, field 2 = 120, field 3 = "INR"
while (pos < bytes.Length)
{
    ulong tag = ReadVarint();
    int field = (int)(tag >> 3); // drop the 3 wire-type bits
    int wire = (int)(tag & 7);
    if (wire == 0) Console.WriteLine($"field {field} (varint) = {ReadVarint()}");
    else { int len = (int)ReadVarint(); Console.WriteLine($"field {field} (length {len}) = \\"{Encoding.UTF8.GetString(bytes, pos, len)}\\""); pos += len; }
}`, mode: "console" },
    react: { starter: `// A PaymentRequest arrived from the order service. Decode it by hand.
const bytes = "08 66 10 78 1A 03 49 4E 52".replace(/ /g, "").match(/../g)!.map((h) => parseInt(h, 16));
console.log("bytes: " + bytes.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" "));
let pos = 0;
function readVarint(): number { let v = 0, m = 1; while (true) { const b = bytes[pos++]; v += (b & 0x7f) * m; if ((b & 0x80) === 0) return v; m *= 128; } }

// Expected: field 1 = 102, field 2 = 120, field 3 = "INR"
while (pos < bytes.length) {
  const tag = readVarint();
  // TODO: this prints the whole tag as the field number. The field number is the tag shifted right by 3.
  const field = tag;
  const wire = tag & 7;
  if (wire === 0) console.log(\`field \${field} (varint) = \${readVarint()}\`);
  else { const len = readVarint(); console.log(\`field \${field} (length \${len}) = "\${new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len)))}"\`); pos += len; }
}`, solution: `// A PaymentRequest arrived from the order service. Decode it by hand.
const bytes = "08 66 10 78 1A 03 49 4E 52".replace(/ /g, "").match(/../g)!.map((h) => parseInt(h, 16));
console.log("bytes: " + bytes.map((x) => x.toString(16).toUpperCase().padStart(2, "0")).join(" "));
let pos = 0;
function readVarint(): number { let v = 0, m = 1; while (true) { const b = bytes[pos++]; v += (b & 0x7f) * m; if ((b & 0x80) === 0) return v; m *= 128; } }

// Expected: field 1 = 102, field 2 = 120, field 3 = "INR"
while (pos < bytes.length) {
  const tag = readVarint();
  const field = tag >> 3; // drop the 3 wire-type bits
  const wire = tag & 7;
  if (wire === 0) console.log(\`field \${field} (varint) = \${readVarint()}\`);
  else { const len = readVarint(); console.log(\`field \${field} (length \${len}) = "\${new TextDecoder().decode(Uint8Array.from(bytes.slice(pos, pos + len)))}"\`); pos += len; }
}`, mode: "console" }
  }
];