Protobuf Encoder & Decoder

Protocol Buffers: The Wire Format Behind gRPC

Protocol Buffers (protobuf) is Google's binary serialization format, designed to be smaller and faster than JSON or XML. It's the default wire format for gRPC and the standard internal RPC format at Google, Netflix, Twitch, Cloudflare, and many other infrastructure-heavy companies. Whenever you debug a microservices system that uses gRPC, you're working with protobuf bytes.

This tool helps you encode and decode those bytes when your generated client code isn't convenient.

Why Protobuf Instead of JSON?

For internal microservices passing millions of messages per second, the size and speed difference matters. Protobuf payloads are typically 30-50% smaller than equivalent JSON and parse 10-20× faster.

The downside: binary is unreadable without tools, and schema-required means more upfront work.

Wire Format Basics

Protobuf encodes a message as a sequence of (field_number, wire_type, value) tuples:

Decoded structure:

• 0a, tag byte: field 1, wire type 2 (length-delimited)
• 05, length: 5 bytes
• 41 6c 69 63 65, value: "Alice" (UTF-8)
• 10, tag byte: field 2, wire type 0 (varint)
• 1e, value: 30 (varint)

With schema (message User { string name = 1; int32 age = 2; }), this becomes:

Without schema, it's a raw structure: {1: "Alice", 2: 30}, readable but unlabeled.

Wire Types

Protobuf has six wire types:

The wire type tells the parser HOW to read the value but not WHAT it means semantically. The schema provides the meaning.

Schema Definition

A .proto file:

The numbers (= 1, = 2) are FIELD NUMBERS, they're the binary encoding's identifier. Once assigned, never change them, or you break wire compatibility. New fields get new numbers. Removed fields' numbers are "reserved" so they can't be reused accidentally.

Forward and Backward Compatibility

Protobuf is famous for compatibility: a new client reading an old server's message ignores fields it doesn't know about; an old client reading a new server's message also ignores unknown fields. Three rules:

1. Never change a field's number. Number 1 must remain meaning-X forever.
2. Never change a field's type (with limited exceptions like int32 ↔ uint32).
3. Never remove a field that's been deployed. Mark it deprecated; reserve the number; add new fields with new numbers.

Follow these and you can evolve schemas indefinitely without breaking clients.

Encoding JSON to Protobuf

To encode, you need:

1. The schema (.proto file).
2. The message type you're encoding.
3. JSON data matching that type's structure.

The tool parses the schema, validates the JSON against it (catching type mismatches), and produces the binary bytes (as hex or base64).

Decoding Protobuf to JSON

To decode, you need:

1. The binary data (hex or base64 input).
2. The schema and message type.

The tool walks the wire format, applies the schema to give semantic meaning, and outputs JSON.

If you don't have the schema, decoding to a "raw protobuf" structure is still possible, field numbers and types, with values shown as best-guess (varints as integers, length-delimited as strings if valid UTF-8 else hex).

Common Pitfalls

Field number changes. Renaming a field in the .proto without changing the number is safe. Changing the number breaks every existing serialized message.

Default vs unset. In proto3, scalar fields have implicit defaults (0, "", false). On the wire, default values may not be written at all. You can't distinguish "explicitly set to 0" from "unset, default is 0." Use optional to track presence explicitly.

Repeated field packing. In proto3, repeated scalar fields are "packed" by default (encoded as a single length-delimited entry instead of repeated tag-value pairs). Older proto2 code may not handle packed; check compatibility.

Map fields. Maps are syntactic sugar for repeated message-with-key-value-fields. On the wire, a map<string, int32> is a repeated message. Order isn't guaranteed; some implementations sort by key, some don't.

Well-known types. Google ships some standard types: google.protobuf.Timestamp, google.protobuf.Duration, google.protobuf.Any, etc. These have special JSON mappings, Timestamp becomes an ISO 8601 string, not a JSON object. Check the JSON mapping spec.

Debugging gRPC: The Real Use Case

When a gRPC call fails or returns unexpected data:

1. Capture the bytes. Wireshark with gRPC dissector, mitmproxy, or grpcurl with --debug.
2. Identify the message type. From the service definition: UserService.GetUser request is GetUserRequest, response is User.
3. Decode with this tool. Paste the bytes, paste the .proto, get JSON.
4. Compare with expected. What field is wrong, missing, or unexpected?

This workflow is invaluable when the server is in a different language than your local environment, or when you only have packet captures from production.

Alternative Tools

• protoc, the canonical Protobuf compiler. protoc --decode=MessageType schema.proto < message.bin decodes from a CLI.
• grpcurl, like curl but for gRPC. Encodes/decodes automatically based on the server's reflection API.
• BloomRPC, Postman, Insomnia, GUI gRPC clients with built-in encode/decode.
• protobufjs (npm), the library this tool uses. For programmatic use in JS apps.
• buf, modern Protobuf tooling for linting, breaking change detection, registry management.

This in-browser tool is for ad-hoc decoding without installing toolchains. For development workflows, use buf and your language's protoc plugin.

Privacy

Protobuf encoding/decoding runs entirely in your browser via the protobufjs library. Schemas (.proto files), JSON inputs, and binary outputs stay in JavaScript memory. Open DevTools Network during use: zero outbound requests. Important because schemas reveal internal service architecture and message payloads sometimes contain live user data, internal IDs, and credentials, none of which should leak.