Plot Your Numbers: Feeding Time Series to VLMs as Images Cuts Tokens and Raises Accuracy

Paper: 2608.07427 Authors: Bhavika Jalli, Nikhil Korati Prasanna, Jayanta Choudhury Categories: cs.AI, cs.PF

The Gap

Two research lines have been running in parallel and mostly ignoring each other.

The first is “LLMs for time series.” Since roughly 2023 people have been serializing numeric sequences into text and prompting a language model — PromptCast-style forecasting, LLMTime, Time-LLM, and the whole family of “just write the numbers in the prompt” approaches. These work surprisingly well, and they have one structural flaw nobody wants to talk about: a floating-point number is a catastrophically expensive thing to tokenize. 0.038271 is not one token. Depending on the tokenizer it’s four, five, sometimes six. Multiply by 96 timesteps, multiply by 24 KPIs, and a single 15-minute window from one cell site becomes tens of thousands of tokens. The paper’s sharpest observation is that this isn’t just expensive — at 24 KPIs it’s infeasible, blowing past the 128K context window that most production models ship with. You either truncate your data or you don’t run.

The second line is energy accounting for inference — the systems-side literature establishing that inference, not training, dominates operational energy at scale, and that inference energy scales close to linearly with input token count. That’s a well-established relationship but it has mostly lived in performance papers, not in modeling papers.

The gap this paper occupies is the intersection: nobody had measured what happens to joules when you change the input modality for numeric data, and nobody had checked whether the cheap modality is also the accurate one. The classical baselines in this domain (ARIMA, LSTM) sidestep the token problem entirely but can’t produce natural-language rationales, and the text-LLM approach that can produce rationales is drowning in tokens.

PROBLEM
[ inference joules ~ input token count ]
[ 24 KPIs x 96 timesteps of floats = 10Ks of tokens ]
[ exceeds 128K context = truncate or fail ]
        |
        v
ASSUMPTION
[ what a model needs from a KPI window is ]
[ shape / trend / spike -- not 7-digit precision ]
        |
        v
METHOD
[ render window as a 2D plot ] --> [ VLM patch encoder ]
[ token budget becomes fixed. decoupled from precision ]
        |
        v
EVIDENCE
[ 3.6x - 10.4x fewer input tokens across 3 VLMs ]
[ 1.8x - 2.5x measured joules per inference ]
[ tuned Llama-3.2-90B-V: +220.7% precision vs text-only ]
[ vs LSTM / ARIMA: +144% on telecom anomaly detection ]
[ Pixtral-12B: 20.6x better J/F1 at mean F1 = 0.82 ]
        |
        v
CONCLUSION
[ for numeric time series pixels are a carrier that is ]
[ both cheaper and more accurate than digits ]

The Increment

One sentence: Before this paper, “plot your time series and show it to a VLM” was a curiosity with a few accuracy results attached; after it, there’s a measured energy ledger behind it plus a hard feasibility argument (past ~24 channels, the text path simply doesn’t fit in a production context window).

Core Mechanism

The pipeline itself is unglamorous, which is fine — the claim is empirical, not architectural. Raw KPI counters stream off 4G/5G cell sites on 15-minute intervals. Instead of serializing the multivariate window into a text block, a renderer draws it: axes, gridlines, one panel or overlaid traces per KPI, saved as a single image. That image goes through the VLM’s vision encoder, which chops it into a fixed grid of patches and emits a fixed number of visual tokens — and this is the whole trick. The patch count depends on image resolution, not on how many data points you drew or how many decimal places each one had. You can draw 96 timesteps or 960; the token bill barely moves.

Those visual tokens get projected into the language model’s embedding space and prepended to a text prompt asking for an anomaly verdict. The decoder answers in natural language, so you keep the thing that made LLMs attractive here in the first place — a rationale, not just a label. For the headline telecom result the authors fine-tune Llama-3.2-90B-Vision on labeled anomalies; the abstract doesn’t specify the adapter scheme, but a LoRA-style tuning of the projector and late layers is the obvious and cheap choice. Energy is measured, not estimated, presumably via GPU power sampling per inference, and folded into a composite metric: joules per unit F1.

The accuracy result is the interesting part and it needs a mechanism story, which the abstract only gestures at. My read: a plot performs an aggressive, lossy, task-aligned compression. It throws away the fifth decimal place — which was never load-bearing for “is this cell degraded?” — and preserves shape, slope, periodicity, and co-movement across KPIs in a spatial layout that a pretrained vision encoder already has good features for. Digit sequences give the model everything and no inductive bias; plots give it less and a better prior.

  4G/5G cell sites (200 cells per 15-min interval)
        |
        v
  +------------------------------+
  | KPI window collector         |  24 KPIs x N steps
  | floats like 0.038271         |
  +------------------------------+
        |
        +---------------------------+
        |                           |
        v                           v
  TEXT PATH (baseline)        VISION PATH (this paper)
  +--------------------+      +------------------------+
  | BPE tokenizer      |      | plot renderer          |
  | one float -> 4..6  |      | axes. gridlines.       |
  | tokens             |      | panel layout. DPI      |
  +--------------------+      +------------------------+
        |                           |
        v                           v
  ~10K - 100K+ tokens         1 image = fixed patch set
  (24 KPIs > 128K limit)      (~1K - 3K visual tokens)
        |                           |
        v                           v
  [ truncate or fail ]        +------------------------+
        |                     | ViT patch encoder      |
        |                     +------------------------+
        |                           |
        |                           v
        |                     +------------------------+
        |                     | projector -> LLM dim   |
        |                     +------------------------+
        |                           |
        +-----------+---------------+
                    v
        +--------------------------------+
        | LLM decoder                    |
        | + fine-tune on telecom labels   |
        +--------------------------------+
                    |
                    v
        [ anomaly verdict + rationale ]
                    |
                    v
        GPU power sampling -> joules / inference
                             -> J per F1

The metaphor: this is a cardiologist reading an ECG, and the paper is an argument about fax machines versus phone calls.

You have a patient’s trace and a specialist in another city. Option A — the text path — is to read the voltage samples aloud down the phone: “zero point zero two three one, zero point zero two four four, zero point zero two five one…” Every digit costs breath. Sixty seconds of ECG at 500 Hz will take you all afternoon, and the specialist’s working memory (the context window) gives out long before you finish, so you skip most of it and hope the interesting part survived. Now add eleven more leads. The call becomes impossible, not merely tedious.

Option B — the vision path — is to fax the strip. The paper cost of a fax page is fixed. One lead or twelve, thirty seconds or five minutes: still one page. That’s the patch encoder. And the specialist reads it faster and better, because a decade of residency trained her eyes on exactly this kind of squiggle — that’s the pretrained vision encoder’s inductive bias. She loses the exact millivolt values in the fax’s grain, but she never needed them; she needed the ST elevation, the rhythm, the shape. The fine-tuning step is her first month at this hospital, learning that this machine’s paper runs a little fast and this population skews a certain way. And the joules-per-F1 metric is the hospital administrator finally asking what each of these two workflows costs per correct diagnosis, a question nobody in the building had been tracking.

The metaphor also tells you where the paper is vulnerable. If the diagnosis genuinely depended on precise millivolts, the fax loses. And a cardiologist is an expensive reader — sometimes a $50 automated rhythm detector is the right call, which is exactly the baseline question I’ll come back to below.

Key Concepts

• The token cost of a number: language model tokenizers were built for prose, and they chop numbers into whatever sub-strings happened to be frequent in text. 0.038271 might split as 0 . 038 27 1 — five tokens for one measurement, and the split isn’t even consistent across similar values. So the model pays five times to receive one number, and then has to reassemble it. Compare a photograph: a 1120x1120 image at 14-pixel patches is a fixed ~6400 patches whether it shows a blank wall or a thousand-point chart. Text token cost scales with your data; visual token cost is a flat rate. That asymmetry is the entire paper.

• Joules per F1: instead of reporting accuracy and efficiency in separate tables, divide. If system A hits F1 = 0.82 for 100 J per inference and system B hits F1 = 0.80 for 900 J, A’s J/F1 is roughly 122 versus B’s 1125 — a 9x efficiency gap that neither an accuracy table nor a latency table alone would show. It’s a crude ratio (it implicitly says a 1% F1 gain is worth exactly 1% more energy, which is rarely true), but it forces energy into the same sentence as quality, which is the point.

• Task-aligned lossy compression: the counterintuitive result here is that throwing information away *improved accuracy. That only makes sense if the discarded information was noise relative to the task. A plot discards precision and preserves geometry. For “did this cell degrade?”, geometry is the signal. This is a general design principle: pick a representation whose losses align with your task’s irrelevancies, and you get compression for free rather than paying for it in accuracy.

Framework Shift

Before (mainstream)                 After (this paper)

  numbers                             numbers
     |                                   |
     v                                   v
[ serialize to digits ]           [ render to pixels ]
     |                                   |
     v                                   v
 token cost =                       token cost =
   precision                          image size
   x length                             (constant)
   x #channels                        |
     |                                 v
     v                            [ fixed patch budget ]
[ grows without bound ]                |
     |                                 v
     v                            fits at any scale.
[ hits 128K wall ]                shape preserved.
     |                            prior of a pretrained
     v                            vision encoder applies
[ truncate. accuracy               |
  falls. joules stay high ]        v
                                  [ fewer joules AND
                                    higher precision ]

From digits to pixels, the core shift is that input cost stops scaling with data volume and becomes a fixed rate you choose in advance — and the compression happens to align with what the task actually needs.

Expert Assessment

Problem choice: The gap is real, but it’s narrower than the abstract implies. The token-inefficiency-of-floats problem is genuine and underappreciated, and the context-window feasibility argument at 24 KPIs is the paper’s most defensible contribution — a capability wall is more persuasive than a cost saving. The energy framing is where I’d push back. “Over 90% of AI operational energy” is a widely repeated figure with soft provenance, and the flagship savings number deserves a unit conversion: 7.2 MJ/day is 2 kWh/day. That’s a household refrigerator. Stated as megajoules it sounds like an infrastructure win; stated as kilowatt-hours it’s about twenty cents of electricity per site per day. The honest version of this paper’s pitch is “we made an infeasible workload feasible, and it also happens to be cheaper,” not “we saved the planet at the telecom edge.”

Method maturity: This is plumbing, and the paper doesn’t pretend otherwise. Rendering time series as images for VLMs already exists in the literature; the increment is the energy ledger and the telecom domain. That’s a legitimate contribution but it’s an empirical-study contribution, and it should be positioned as one. The bigger issue is a simpler approach hiding in plain sight: patch-based numeric tokenizers. PatchTST and Chronos-style models already compress numeric sequences into a small number of learned tokens, achieving the same decoupling of token count from series length *without the detour through a rasterizer. If your goal is fewer tokens per KPI window, a purpose-built numeric tokenizer is the direct route and pixels are the scenic one. The paper’s implicit answer is presumably “but then you lose the pretrained VLM and its natural-language output,” which is a fair answer — I’d just want it argued rather than left unstated.

Experimental integrity: This is where I’d spend my review. “+220.7% precision over its text-only counterpart” is the number that worries me. A 3.2x precision gain from changing input format alone is not a plausible modality effect; it’s the signature of a broken baseline. Two likely mechanisms: (a) the VLM was fine-tuned and the text-only model wasn’t, in which case the comparison measures fine-tuning, not modality; and (b) at 24 KPIs the text model *had to be truncated to fit the context window, so it was answering with a fraction of the data. Both are noted in the abstract, which makes the comparison self-undermining — if the text path is infeasible, then say it’s infeasible and stop there. Reporting a percentage gain against a crippled baseline converts a clean feasibility argument into a suspicious accuracy claim. Similarly, “20.6x improvement in J/F1” needs its denominator spelled out: against text-only Pixtral? Against LSTM? Against the truncated variant?

And the energy comparison has a conspicuous hole. The accuracy comparison against LSTM and ARIMA (+144%) apparently doesn’t come with an energy comparison — but a small LSTM runs anomaly detection for microjoules on a CPU, three to four orders of magnitude below a 90B VLM. If joules are a first-class constraint, the most energy-efficient system in the paper is almost certainly the LSTM baseline, and the paper’s own metric would say so. The defensible claim is conditional: given that you want an LLM in this loop for its reasoning and explanations, images are the cheap way to feed it. That framing survives scrutiny. The unconditional “energy-efficient modality” framing doesn’t.

Writing quality: The abstract is a wall of ratios — nine numeric claims in eleven sentences — and ratio density is usually inversely correlated with methodological detail. Two sections almost certainly need more room than they got. First, the energy measurement methodology: what instrument, sampled at what rate, batch size, prefill versus decode attribution, whether idle power was subtracted. Every joule claim rests on this and it’s the easiest thing in the paper to get wrong. Second, plot rendering as a design space. Axis scaling, DPI, gridline density, color mapping, and panel layout are now hyperparameters that directly affect accuracy, and an ablation over them would be the paper’s most reusable asset. If I could force one rewrite, it would be replacing the percentage-gain claims with a clean fine-tuned-versus-fine-tuned comparison at a channel count where both modalities fit.

Verdict: borderline — the feasibility argument and the measured energy numbers are worth publishing, but the headline accuracy gains rest on baselines that appear structurally disadvantaged, and the energy framing omits the small-model comparison that its own metric demands.

Takeaways

Concrete things worth stealing:

Fixed-cost encoders as a compression channel. The transferable insight isn’t “plots are good.” It’s that any modality with a fixed-size encoder — images, audio, learned patch tokenizers — decouples prompt cost from data volume. If you’re stuffing oversized structured input into a context window (logs, tabular data, dependency graphs, traces), ask whether there’s a fixed-rate encoder that can carry it. The flat-rate-versus-metered distinction is the reusable frame.

Lead with feasibility, not cost. The strongest sentence in this paper is that at 24 KPIs the text representation doesn’t fit, full stop. Cost savings invite haggling; capability walls don’t. If you’re pitching an efficiency change internally, find the threshold where the old approach stops working at all and lead with that.

Put joules in your eval table. Adopt the joules-per-quality ratio even if the exact ratio is crude. Once energy sits in the same table as F1, architecture conversations change — and note that this paper’s own metric would have flagged the LSTM question if applied consistently. A metric that only ever flatters you isn’t being applied honestly.

Rendering is now prompt engineering. If you go down this path, your matplotlib defaults are model hyperparameters. Fix DPI, axis normalization, gridlines, and color mapping explicitly, version them, and ablate them. Silent renderer changes will look like model drift and you will lose a week finding it.

Always price the small model. The discipline this paper lacks is the one to keep: before claiming an efficiency win for a large model, compute joules-per-quality for the boring 5M-parameter alternative. Sometimes the answer is that you don’t need the large model at all, and it’s better to learn that from your own table than from a reviewer’s.