zeroshot research blog

A dark factory for time series collection

The internet is full of valuable time series data, but it comes in inconsistent formats and is often not directly usable. We built a system that automatically generates ideas for data sources, locates suitable recordings and checks their usability, all within a single zeroshot graph.

A microphone on Mars and the flight muscles of a fruit fly both produce time series. Training and evaluating time series forecasting models across such different processes calls for recordings with varied time scales and relationships between measured variables. Foundation models are one use for this kind of breadth.

The internet is full of such observations. Scientific instruments deposit recordings in public archives, while experiments leave behind supplementary files and spreadsheets. Operational systems produce measurements for their own purposes. Much of this material sits outside the collections people usually reach for when training a forecasting model.

Finding it does not give you a usable dataset. The files follow the conventions of a particular instrument or experiment. A timestamp might live in a separate header, a binary format might need a custom reader, and the meaning of a column might only become clear after reading the accompanying paper.

A diverse training collection needs machinery that can work through those differences. With zeroshot, we can turn that work into an ongoing process that looks for sources, investigates their recordings and checks whether they qualify.

Define what qualifies

A usable series needs at least 1,000 observed timestamps in a coherent run, with a known clock and documented units and quality handling. We prefer related variables on a shared clock and require permission for the intended use. Interpolation cannot supply missing observations, and extra channels cannot make a recording longer.

Read the shared validation rules

A summary of the common requirements behind the node instructions.

  1. Use original observations. Retrieve and decode the source recording. Establish how the measurements were acquired and processed. Synthetic values, feature tables and hypothetical faster sampling cannot qualify as original observations.
  2. Count a coherent run. Require at least 1,000 finite, observed timestamps on a known native clock. Inspect gaps and duplicates, and report elapsed duration. Extra channels and interpolated values do not increase the observed timestamp count.
  3. Preserve meaning and provenance. Identify each variable and its source-defined unit or instrument scale. Record quality flags, exclusions and missing values, along with the acquisition route and original file hashes. Never invent a calibration.
  4. Check permission and distinctness. Establish permission for the intended research use. Compare the measurement process with prior findings and published source families. Renamed stations or channels alone do not establish additional information.
  5. Require independent checks. Freeze the producer’s findings before review. A separate implementation must reproduce an accepted count, while other reviewers check methods and distinctness. Preserve critical unknowns and original reviews. Accept a source only when all required criteria pass.
  6. Bound the work and retain its outcome. Use authorized requests within shared resource limits. Record scientific rejections and stop on unresolved operational failures. Preserve compact evidence and acquisition recipes before deleting temporary observations after review.

Turn the research task into a loop

Each source can require a different small program. Coding agents can investigate documentation and implement a reader for the format they find. The graph gives that flexible work a fixed order and a set of checks before a source enters the accepted inventory.

Discovery starts with parallel proposals. An explorer looks for public sources of original observations. A synthesizer connects measurement mechanisms across domains, while a challenger looks for less familiar alternatives and obstacles the other proposals may have missed. The selector compares their findings and commits to one acquisition plan.

A probe then retrieves and decodes the selected recording within the allowed budget. When a format is difficult, it can investigate credible readers and documentation during that attempt. Its findings become fixed inputs for independent review.

The self-contained source-validation graph Start checks lead into a repeated investigation. Three scouts feed source selection and readiness. A committed source is probed, frozen and reviewed independently before its result is recorded. If the selector cannot commit to a source, an operational audit checks the attempt before closure. Continuation starts the next source or stops at a target, hold or limit. Select a node to read its instructions.
Click a node to read its instructions. References to the shared validation rules open the summary above. The return path starts another investigation after the previous attempt has closed.

The evidence reviewer reproduces the count and clock facts with a separate implementation. The method reviewer checks recording semantics and whether the handling of gaps, units and quality is justified. Originality review compares the acquired information with published source families and any sources already accepted by the workflow. An agent proposing an interesting source cannot accept it on that basis alone.

Once an investigation has closed, zeroshot routes the work back to discovery. A rejected source leaves its reason in the record, so later searches can use what the machinery has already learned. If no proposal is ready, an operational audit checks the attempt before the next handoff.

The return path is what makes this a dark factory. Routine discovery and checking can proceed without a person dispatching each task or carrying findings between agents. Budgets and stop conditions bound the loop. A scientific rejection can lead to another source, while an unresolved operational failure stops the run.

Accepted sources leave an acquisition recipe and a record of the observations that passed review. Temporary recordings remain available for the independent checks, then get deleted after the result and its provenance are durable. The retained recipe supports acquiring the data again for training or evaluation.

What a diverse collection looks like

These recordings illustrate the range of sources the machinery can work with. Each brings its own interpretation of time and its own measurement semantics.

A microphone on Mars

SuperCam microphone files from the Perseverance mission contain waveforms in uncalibrated digital counts. The sample clock comes from the instrument metadata. Decoding the file gives a numerical series while preserving the original units, without calling its amplitude calibrated sound pressure.

A fruit fly in tethered flight

A Drosophila experiment pairs a flight-muscle electrode with a laser wingbeat detector in a deposited Spike2 file. Both channels report voltage on the same clock. Keeping the waveforms together preserves two views of the flight process that a table of average wingbeat frequencies would lose.

Pressure inside a chemical garden

A chemical garden grows solid structures from substances dissolved in a liquid. An experiment on one of its walls records pressure relative to the atmosphere, in kilopascals. The useful series is the sensor’s pressure record, with the experiment’s clock and measurement conditions retained.

A bead in an optical trap

An optical-tweezer experiment with a driven trap centre records photodetector voltages. These are detector outputs, rather than calibrated bead positions. The file keeps three signals on a shared clock: two detector directions and summed light.

Beyond time series collection

Zeroshot is well suited to ongoing tasks that need LLM judgment and should run without someone supervising every step. The same structure can support other research and processing work whenever its results can be checked against evidence.

In a single growing agent session, the same assumptions can influence what the agent investigates and how it judges its own output. If an early mistake shapes later choices, that bias can compound across the work.

Zeroshot makes acceptance a separate step with its own session and evidence requirements. Reviewers can challenge a proposed result before it becomes an input to later decisions, while the graph keeps routine execution moving within its operating limits.

What is zeroshot? →