What Hides Under the Clock?

A choice can occupy a piece of silicon

In the 1990s, I soldered transistors and integrated circuits, drew large schematics on A1 sheets, etched circuit boards, and wired components directly together. A NAND gate, comparator, multiplexer, counter, or Schmitt trigger seemed like an arrangement of switches. That intuition was useful. What I had yet to appreciate was how much could be built by arranging the switches and their memory carefully.

Take a two-input multiplexer. It has two data inputs, A and B, and a selection input, S. Its output follows A when S is one and B when S is zero:

assign out = select ? a : b;

The hardware meaning of this expression is wonderfully direct. The circuit exists continuously. A change at the selected input propagates to the output after the circuit's delay. Changing the selection changes which input can influence the output. There is no instruction pointer that has to revisit this line to make the multiplexer respond.

Fix A=0 and B=1, and the output is NOT S. Fix B=0, and the output is S AND A. Fix A=1, and the output is S OR B. A multiplexer can implement each of these familiar Boolean operations. Cascading multiplexers lets one input choose between functions of the other inputs, which is the same decomposition behind a binary decision tree.

That gives us an excellent starting point for computation: inputs influence switches, switches influence other switches, and the resulting pattern appears at the outputs. But a combinational network, however elaborate, only transforms its present inputs. To remember a previous input, count a packet's bytes, or return from a function call, the machine needs state.

Give the computation somewhere to live

Programming introduced me to a much larger vocabulary: Turing machines, actors, cellular automata, lambda calculus, SKI combinators, interaction nets, processes, threads, recursion. I spent the next twenty years exploring their differences. The models that attracted me most allowed activity in many places and allowed several pieces of work to be in flight together.

Suppose a program applies three transformations to a stream of samples. A sequential implementation can reuse one execution resource: perform the first operation, then the second, then the third. Its working state records where it is and the intermediate values it will need later. A spatial implementation can give each transformation its own circuit. Now the intermediate values have physical destinations, and the connections between those destinations express the dependencies.

Two engineering choices become visible. We can duplicate resources so that several values are processed side by side. We can also place storage between transformations so that a new value enters the first stage while an older one moves through the second. The latter is pipelining. The choices can be combined.

This is a much more productive question than whether a language is sufficiently “high level.” Where are the values? Which operators exist simultaneously? Which operators are shared? What happens when the next value arrives before the previous one has finished? Those decisions determine throughput, storage, latency, and how a system behaves under load.

An FPGA makes these choices unusually tangible. You describe a network of logic and storage, and the tools map it into the device. A loop in the source may describe repeated hardware, or it may participate in a design that reuses hardware across cycles. The source text alone does not tell you which cost you are paying; the resulting circuit does.

Mark Allyn's light wand brought me back to hardware

My route back into hardware began with my friend Mark Allyn, a Bellingham artist and maker who turns light into things people can wear, carry, and dance with. Mark is one of Bellingham's stars. In a local discussion of the city's beloved “Belebrities,” residents thank him for what he brings to the community and describe the pleasure of seeing his cheerful outfits. The affection makes sense: Mark makes things, shares what he learns, and brings his creations out into the world.

His personal website, Mark Allyn Artistic Journal, puts the wand in the context of a lifetime of making. He remembers discovering persistence of vision as a child by moving a toy flashlight across his bedroom ceiling. That fascination grew into illuminated clothing, metal and glass sculptures, jewelry, and handmade rainwear that lets the clothing beneath it remain visible. His site also shares Blender models of buildings from his childhood and from Bellingham. Sewing, welding, electronics, and animation all serve the same generous purpose: making something in his imagination available for other people to experience.

The wand has its own public history in Mark's words. On August 28, 2019, he announced My First FPGA Project: an FPGA controlling programmable RGB LEDs to create persistence-of-vision patterns. His October 3 follow-up, FPGA Project For Light Wand, describes colored sinusoidal patterns appearing as he waves the wand through the air. He published the Verilog source and project documentation and invited others to use them with credit. The repository includes photographs, a written explanation, and the RTL itself. Mark made the path into his work available to anyone curious enough to follow it.

The physical idea is beautiful. Move a strip of LEDs through the air while changing their colors, and successive moments of light appear at successive positions along the movement. The changing light and the moving wand together form the pattern. Timing becomes visible: changing when an LED lights changes where its contribution appears. Mark's artistic goal gave the hardware problem a purpose you could see.

Mark also asked me to include a detail he shared in private correspondence: the FPGA-driven wand worked so well that he got kicked out of Rumors Cabaret in Bellingham. For Mark, the circuit belonged out in the world, making light art in motion.

Mark had tried Raspberry Pi and Arduino approaches and moved to an Artix-7 FPGA. I bought the same Arty board and LED strip, installed Vivado on my Debian laptop, and followed his work until the strip lit up. His engineering and his willingness to document it brought me back to the bench. You can explore more of his work on GitHub as maallyn and follow Mark Allyn on LinkedIn.

Then the available hardware pulled me further in. The Lattice IceStick, with an iCE40HX1K, led to the open IceStorm documentation, the bitstream format, and the C++ source of arachne-pnr. I bought iCE40LP384 parts and learned to solder their QFN packages with hot air. A microscope later made working with an iron practical as well.

Years of Python and JavaScript had made me impatient with slow iteration. I wanted to change a design and see the result quickly. The Arty's Ethernet connector suggested a direct route from my laptop to the hardware. I wrote ARP and UDP logic and used the board to program an adjacent iCE40 through its configuration pins.

The Ethernet-to-iCE40 reprogrammer grew from this work. A packet could arrive, its bytes could be received and stored, and a configuration sequence could deliver those bytes to another chip. Each activity had its own state and timing. Building the connection forced me to understand what the Verilog actually meant.

Two assignments can describe one simultaneous change

I printed roughly fifty pages of my Verilog and read them closely. The code seemed to say everything twice. There were current values and next values, levels and edges, logic that continuously derived an answer and logic that decided when to preserve it. My software instincts wanted a shorter sequence of actions.

Consider two registers. Ignore reset for a moment and look only at their update rule:

always_ff @(posedge clk) begin
    a <= b;
    b <= a;
end

At the active clock edge, each register takes the other's previous value. If a held 3 and b held 9, afterward they hold 9 and 3. The nonblocking assignments describe updates from the old state. Reading them as ordinary immediate software assignments would produce the wrong mental picture.

The wires from the registers' outputs already carry the old values before the edge. Each receiving register has a candidate value waiting at its input. The clock causes the receiving elements to capture those inputs, subject to their timing requirements. The two changes are part of one logical state transition even though, physically, every clock and data path has delay.

Write the entire stored state as q and the inputs as x. A synchronous machine can be described as q_next = F(q, x). Between edges, the combinational circuit computes F. At the edge, the registers capture its result. An output may depend on the stored state alone, or on stored state and current inputs; that is the familiar Moore/Mealy distinction.

Feedback now has a precise meaning. A register's output returns through logic to help determine its next input. The storage separates one transition from the next. A wire loop without that discipline has different behavior: it may form a latch, oscillate, race, or fail to settle. Drawing an arrow back to the beginning does not by itself create the orderly iteration we expect from a software loop.

State machines give each cycle a purpose

David Harel's 1987 paper, Statecharts: A Visual Formalism for Complex Systems, helped me organize the control side. Hierarchy lets us describe a larger activity without repeating every internal detail. Concurrent regions let separate activities maintain their own progress. Transitions explain which events change that progress.

A packet receiver offers a concrete example. It might wait in IDLE, recognize a header in HEADER, collect bytes in PAYLOAD, and report completion in DONE. A payload-byte counter records progress inside one of those states. The state selects which incoming bytes matter and which register enables should be asserted.

The counter does not need to know the entire protocol. The controller does not need to implement the arithmetic of incrementing a counter. Their interface is enough: an enable says “count this byte,” and a comparison says “the expected number has arrived.” That relationship can be repeated through a hierarchy of modules.

For four states, binary encoding can use two flip-flops. One-hot encoding uses four, with one active bit identifying the state. That extra storage can simplify the logic that tests a state or computes a transition. It often suits FPGA resources well, though the best choice depends on the actual machine and its implementation.

This was the missing connection to the distributed software I already knew. A component can wait for an event while another component keeps working. A request can influence a neighbor's next transition. The implementation does not require a single giant controller to visit every participant one at a time. The meaning of an interaction still has to be designed: which cycle presents a request, which cycle accepts it, and what keeps the request from being lost or counted twice?

The clock buys a bounded interval for the work

I initially resisted the clock. It looked like an imposed global schedule in a world I wanted to make reactive. Trying asynchronous control and handmade edge detection made the timing relationships more visible. Signals still took time to travel. Feedback still needed an ordering rule. A circuit still needed a way to decide that an input was ready to use.

In a synchronous region, the clock gives us a useful interval. A launching register changes its output after an edge. The change propagates through logic and routing. The destination must receive a stable value early enough before its capture edge to meet setup time, and the value must remain stable long enough afterward to meet hold time.

For a simplified one-cycle path, the setup budget includes clock-to-output delay, maximum logic and routing delay, receiving setup time, and allowance for clock uncertainty and the relevant skew. Suppose a 100 MHz clock gives us 10 ns. An illustrative budget might be 0.4 ns to leave the first register, 6.8 ns through logic and routing, 0.6 ns of setup, and 0.7 ns of timing allowance. The total is 8.5 ns, leaving 1.5 ns of slack.

Those are example numbers, not a specification for a particular FPGA. The useful point is the accounting: the whole path must fit. Move the same logic farther apart and the routing delay can consume the margin. Increase the clock frequency and the available interval shrinks. Insert an appropriately placed pipeline register and each resulting stage gets its own interval.

Hold timing asks a different question: can the new value arrive too soon and disturb the value being captured? Slowing the clock does not generally repair a same-edge hold violation. Minimum delays and clock arrival relationships matter as well as maximum delays.

Timing closure is the work of making the placed and routed circuit obey these relationships across the specified operating conditions. The benefit is enormous. Once the conditions hold, we can reason about the region in state transitions instead of tracing every intermediate electrical change through the chip.

A clock boundary is a communication boundary

A button, sensor, radio comparator, or independently clocked neighbor has no obligation to change at a convenient point in our local cycle. If an input changes close to a receiving edge, the first flip-flop can become metastable: its output can take longer than usual to resolve to a usable logic level.

For a slowly changing single-bit level, two or more receiving-clock registers in series are a standard solution. The first samples the asynchronous input. The following stage samples later, allowing resolution time. Device characteristics, event rate, receiving frequency, placement, and available settling time affect reliability. This is why the synchronizer is a deliberately implemented structure, not just two convenient variables.

A short pulse needs more thought. It can occur entirely between sampling edges and never be observed. A request/acknowledge handshake can hold the event until it is accepted; a toggle scheme can encode an event as a persistent change, provided the event spacing and receiver protocol support it. The application determines what must be preserved.

Reset is part of the same design. Where asynchronous assertion is required, release is commonly synchronized separately to each receiving clock. Otherwise different state elements may leave reset on different cycles, damaging the very initial condition reset was meant to establish. The exact reset strategy follows the device and the system's startup requirements.

A multi-bit value makes the problem harder. Suppose a binary counter changes from 0111 to 1000. Four bits change. Synchronizing each bit independently can expose a combination that was never a valid counter value at the source.

An asynchronous FIFO gives a stream a more complete interface. The writer stores data using its clock; the reader retrieves it using another. Read and write pointers track progress. A common implementation crosses Gray-coded pointers, whose adjacent values differ in one bit, and uses the synchronized progress to decide full and empty. Physical constraints must also preserve the intended pointer-crossing behavior. The payload is held in memory; it is not reconstructed by independently synchronizing every payload bit.

The FPGA vendor's CDC guidance helps select an implementation suited to the kind of information crossing. The underlying design question remains straightforward: are we sending a level, an event, a coherent word, or a stream? Each has different obligations.

A FIFO can absorb bursts and short-term differences in progress. It cannot absorb an indefinitely faster producer with finite memory. The surrounding protocol needs backpressure, a rate guarantee, or an explicit policy for overflow. Buffering makes waiting manageable; it does not make storage infinite.

Local clocks let the larger system keep moving

The breakthrough for me was accepting that synchronous reasoning could be local. My familiarity with distributed systems and relativity had already made me suspicious of one universal “now.” Digital engineering gives that intuition a concrete form: clock domains can maintain their own timing, and interfaces can carry information between them.

Daniel Chapiro's 1984 work on globally asynchronous, locally synchronous systems gave this arrangement a name: GALS. Build synchronous regions that are practical to time and implement. Connect them with interfaces that handle independent progress.

Imagine an acquisition block producing samples at its own rate, a processing block transforming them, and a host interface transferring results in bursts. An input FIFO can bridge acquisition and processing; an output FIFO can bridge processing and the host. Each region has an internal schedule. The FIFO protocols determine when data is available and when space is available.

That structure makes several kinds of reasoning possible at once. We can close timing within a region. We can check that a crossing preserves data and ordering. We can calculate whether the buffers cover the expected bursts. We can define what the source must do when the destination stops accepting work.

This applies at different scales, but the implementation details belong to each scale. An on-chip FIFO, an FPGA-to-FPGA link, and a network connection use different mechanisms. What they share is the need to preserve meaningful progress across an interface. The clock-region article follows that concern into placement and timing.

Datapath and control belong together

I had also resisted the separation of datapath and control logic. It seemed like another unnecessary division imposed on a computation that should describe itself. The packet receiver had already supplied the answer: moving and transforming values is one part of a job; deciding which movement or transformation counts is another.

A datapath may contain adders, multipliers, shifts, comparisons, registers, and multiplexers. Control selects operands, enables captures, identifies valid results, and decides when the next operation may begin. These are connections between circuits. “Control” does not imply a software thread supervising every gate.

For example, an iterative divider reuses arithmetic over several cycles and records its progress in registers. A pipelined multiplier distributes work over stages and may accept another operand pair before the preceding product emerges. Both need rules for identifying which inputs were accepted and which outputs are meaningful. Their storage and schedules are different because their resource choices are different.

Even z = a × x + y contains an architectural decision. If the multiplication takes several pipeline stages, the corresponding y value must wait or travel through matching storage so that the addition combines the correct operands. For floating-point fused multiply-add, the internal precision and final rounding are also part of the operation's meaning. A formula names the relationship; the circuit must preserve that relationship over time.

The same approach scales into a system: buffered DSP pipelines for regular streams, state machines for control, and soft CPU cores where sequential or irregular tasks make them useful. The interfaces join them. We can choose the structure that suits each task and still reason about the composed machine.

A pipeline turns waiting time into useful overlap

Consider a three-stage circuit that scales a sample, adds a fixed bias, and compares the result with a threshold. Label successive samples A, B, C, and D. An input register launches A at edge 0. After one interval, the first stage captures A's scaled value. Subsequent edges advance it while admitting more work:

At edge 3, A has reached the output. At edge 4, B follows. The latency from input-register capture to output capture is three cycles. The initiation interval—the spacing between accepted samples—is one cycle. At an illustrative 100 MHz, that is 30 ns of latency and, once filled, 100 million results per second, assuming each stage meets timing and the interfaces sustain the stream.

The labels in the table are as important as the arithmetic. At any instant, the stages contain values from different samples. If the bias were an input that varied with each sample, it would need to travel with its own sample through the appropriate delays. A valid bit, channel identifier, or packet marker often needs the same treatment.

Now let the destination stop accepting results. A fixed-flow pipeline needs a guaranteed drain or enough output buffering for the pause. An elastic pipeline uses valid/ready relationships and storage so that occupied stages can hold their values and propagate backpressure. In either design, the rule must preserve the association between data and its meaning.

This was the concurrent, reactive behavior I had been searching for. The scale, bias, and comparison circuits all exist. A change at each stage's inputs starts that stage's work. Registers preserve the intermediate results. The next sample does not need to wait for the previous sample to finish the entire computation.

Four flip-flops make finite state impossible to ignore

A circuit I made in Logisim brought the state model into sharp focus: four D flip-flops and an XOR feedback function, arranged as a linear-feedback shift register. The output looked irregular, but I could watch every bit of memory that generated it.

Four bits have sixteen possible patterns. A maximal-length four-bit XOR LFSR visits fifteen nonzero states before repeating; the all-zero state is a separate fixed point. The apparent irregularity comes from the order of the states. It does not create additional states beyond the memory available.

I encountered the same idea in a more surprising form while running programs for a very small single-instruction processor that could modify its own code. The deck points to BitBitJump: an instruction identifies a source bit, a destination bit, and the address of the next instruction. Copying a bit can alter the memory from which later instructions will be read.

When I explored the programs that fit in the small memory, the recurring sequences became impossible to dismiss. The reason is simple enough to state precisely. A closed deterministic machine with N bits of total state has at most 2^N states. After more than that many observations, some complete state must have occurred before. Because the transition rule is deterministic, the same complete state has the same successor. From that point onward, the future repeats.

The word complete matters. For a processor, include the program counter, registers, memory, and any other state that affects the next step. A repeated RAM image alone need not mean a repeated machine state. The path can also have a transient before entering a cycle. A halted state can be represented as a state that keeps returning to itself.

An open machine receives new inputs, so its next state depends on more than its internal history. A keyboard event, a network packet, or a fresh sample can change the trajectory. This gives an interface real conceptual weight: it is where information from outside the machine enters its evolution.

The practical lesson is liberating. Complicated behavior can be understood through stored state, transition rules, and inputs. For a fixed physical machine, all three can be made concrete. General models of computation extend the available memory when the problem requires it; a particular chip gives us a particular finite implementation.

Reactive programming needs somewhere to put the value

Once I could see circuits this way, familiar software arguments became easier to untangle. A reactive dependency says that a change upstream may require work downstream. The implementation still needs to decide where the value waits, who initiates a transfer, and what happens if the receiver is busy.

In a push arrangement, a producer presents work when it is available. A consumer must accept it, buffer it, or signal that it cannot yet proceed. In a pull arrangement, demand from the consumer initiates delivery. A buffered stream combines both ideas: production fills available storage, while consumption creates space and permits further production.

A same-clock valid/ready interface makes this explicit. The producer presents a value and asserts valid. The receiver asserts ready when it can accept the value. A transfer occurs on a clock edge when both are asserted. While valid is asserted and ready is low, the producer holds the offered value stable. These rules tell us exactly which event advances each participant.

Software queues, actor mailboxes, and streaming systems face comparable resource questions. Their runtime may implement the storage and scheduling for us, but the obligations remain: retain work until it is accepted, preserve required ordering, associate responses with requests, and decide how finite capacity affects admission.

Recursion has a physical account, too. If a call must return to unfinished work, some state must preserve that continuation and any values it will need. A stack is one arrangement of that storage. A circuit with multiple active invocations needs corresponding resources or a schedule for sharing them. Naming the abstraction does not eliminate its state.

What finally impressed me about sequential logic was its breadth. It supports parallel work because many circuits occupy different places. It supports overlapping work because registers and buffers preserve progress. It supports reactive behavior because connected logic responds to changing inputs. It supports larger distributed systems because local machines can communicate through designed interfaces.

The clock lets us use all of this with a tractable account of time. Beneath each edge is a network of choices; between edges, signals propagate; across edges, state carries history forward. Once those mechanisms become visible, a chip stops looking like a mysterious box that executes code. You can see where the computation lives, how it moves, and what another part of the machine is free to do while it waits.

If you are designing a stream processor, a reconfigurable system, or a piece of software whose concurrency has become difficult to explain, draw its storage and transfers. Mark what each participant remembers, when it may advance, and how it knows its neighbor is ready. That drawing is often the beginning of a much better architecture. Bring me the system you want to build, and let's work through it.