Also fixed from earlier suggestions and feedback as noted before:
- Data zoom slider bug has been fixed (no longer snapping to the left or right)
- Idle CPU usage bug (added user controls along with more clarity to 1M point benchmark)
13 hours on the front page, 140+ comments and we're incorporating feedback as it comes in.
This is why HN is the best place to launch. Thanks everyone :)
Update: Patched idle CPU usage while nothing is being rendered.
One thing to note: I added a toggle to "Benchmark mode" in the 1M benchmark example - this preserves the benchmark capability while demonstrating efficient idle behavior.
Another thing to note: Do not be alarmed when you see the FPS counter display 0 (lol), that is by design :) Frames are rendered efficiently. If there's nothing to render (no dirty frames) nothing is rendered. The chart will still render at full speed when needed, it just doesn't waste cycles rendering the same static image 60 times per second.
Blown away by all of you amazing people and your support today :)
Interesting idea - I haven't explored wavelet-based approaches but the intuition makes sense: decompose into frequency bands, keep the low-frequency trend, and selectively preserve high-frequency peaks that exceed some threshold.
My concern would be computational cost for real-time/streaming use cases. LTTB is O(n) and pretty cache-friendly. Wavelet transforms are more expensive, though maybe a GPU compute shader could make it viable.
The other question is whether it's "visually correct" for charting specifically. LTTB optimizes for preserving the visual shape of the line at a given resolution. Wavelet decomposition optimizes for signal reconstruction - not quite the same goal.
That said, I'd be curious to experiment. Do you have any papers or implementations in mind? Would make for an interesting alternative sampling mode.
I don't. I just remember watching a presentation on it and it always struck me that wavelets are an incredibly powerful and underutilized technique for data reduction while preserving quality in a quantifiable and mathematically justifiable way.
I don't have any papers in mind, but I do think that the critique around visual shape vs signal reconstruction may not be accurate given that wavelets are starting to see a lot of adoption in the visual space (at least JPEG2000 is the leading edge in that field). Might also be interesting to use DCT as well. I think these will perform better than LTTB (of course the compute cost is higher but there's also HW acceleration for some of these or will be over time).
No, FFT is perfectly information preserving by definition. Thats why there’s an inverse FFT operation that restores the original signal without any loss (well, modulo accumulated floating point error when working in the discrete instead of symbolic space).
Really appreciate you taking the time to look, Leon - uPlot has been a huge inspiration for proving that browser charts don't have to be slow.
Both points are fair:
1. LTTB peak elimination - you're right, and that PR is a great reference. For the 1M demo specifically, sampling is on by default to show the "it doesn't choke" story. Users can set sampling: 'none' for apples-to-apples comparison. I should probably add a toggle in the demo UI to make that clearer.
2. Idle CPU - good catch. Right now the render loop is probably ticking even when static. That's fixable - should be straightforward to only render on data change or interaction. Will look into it.
Would love your deeper dive feedback when you get to it. Always more to learn from someone who's thought about this problem as much as you have.
Blind sampling like this makes it useless for real-world statistics of the kind your users care about.
And column-oriented data is a must. Look at Rlang's data frames, pandas, polars, numpy, sql, and even Fortran's matrix layout.
Also need specialized expicitly targetable support for Float32Array and Float64Array. Both API and ABI are necessary if you want to displace incumbents.
There is huge demand for a good web implementation. This is what it takes.
Great suggestion - density mapping is a really effective technique for overplotted data. Instead of drawing 1M points where most overlap, you're essentially rendering a heatmap of point concentration. WebGPU compute shaders would be perfect for this - bin the points into a grid, count per cell, then render intensity. Could even do it in a single pass. I've been thinking about this for scatter plots especially, where you might have clusters that just look like solid blobs at full zoom-out. A density mode would reveal the structure. Added to the ideas list - thanks for the suggestion!
Drawing lots of single pixels with alpha blending is probably one of the least efficient ways to use the rasterizer though. A good compute shader implementation would be substantially faster.
At 1M points it hardly makes a difference. Besides, 1 point -> 1 pixel mapping is good enough for a demo, but in practice it will produce nasty aliasing artifacts because real datasets aren't aligned with pixel coordinates. So you have to draw each point as a 2x2 square at least with precise shading, and we are back to the rasterizer pipeline. Edit: what actually needs to be computed is the integral of the points dataset over each square pixel, and that depends on the shape of each point, even if it's smaller than a pixel.
Aren't we at petaflops now with GPUs? 1M or even 1G points should be no issue if it renders to a framebuffer and doesn't go through mountains af JS framework rubbish followed by mountains of GTK/Qt/.NET rubbish.
Not true. Fill rate and memory speed is still a huge bottleneck. The issue is not “rubbish” but memory speed. It is almost always memory speed, cache, ram, disk etc.
There is this misconception that if one uses js or c# to tell a gpu what to do it is somehow slower than rust. It only is if you crunching data but moving memory to the gpu and telling gpu to crunch is virtually identical.
Most consumers dont have that and at 60 fps you are already maxing it out and more assuming os is doing nothing else. Bandwidth even on gpus is still the bottleneck.
Even then, when u write to a framebuffer directly in the gpu if the locations of the points are not contiguous you are thrashing. Rendering points very fast is still very much about reducing the data set down to bypass all the layers of memory walls.
That works if more overdraw = more intensity is all you care about, and may very well be good enough for many kinds of charts. But with heat map plots one usually wants a proper mapping of some intensity domain to a color map and a legend with a color gradient that tells you which color represents which value. Which requires binning, counting per bin, and determining the min and max values.
Wait, does additional blending let you draw to temp framebuffers with high precision and without clamping? Even so you'd still need to know the maximum value of the temp framebuffer though.
That's what EXT_float_blend does. It's true, though, that you can't find the global min/max in webgl2. This could be done, theoretically, with mipmaps if only those mipmaps supported the max function.
Couldn't you do that manually with a simple downscaling filter? I'd be very shocked if fragment shaders did not have a min or max function.
Repeatedly shrinking by a factor of two means log2(max(width, height)) passes, each pass is a quarter of the pixels of the previous pass so that's a total of 4/3 times the pixels of the original image. Should be low enough overhead, right?
Sure, that will work, but it's log2 passes + temp framebuffers. As for overhead, I'm afraid it will eat a couple fps if you run it on every frame. In practice, though, I'm not sure that finding the exact maximum is that valuable for rendering: a good guess based on the dataset type will do. For example, if you need to render N points that tend to congregate in the center, using sqrt(N) as the heuristic for the maximum works very well.
That's a cool project! Just checked out the workbench. I should be upfront though: ChartGPU is currently focused on traditional 2D charts (line, bar, scatter, candlestick, etc.), not graph/network visualization with nodes and edges. That said, the WebGPU rendering patterns would translate well to force-directed graphs. The scatter renderer already handles thousands of instanced points - extending that to edges wouldn't be a huge leap architecturally.
Is graph visualization something you'd want as part of ChartGPU, or would a separate "GraphGPU" type library make more sense? Curious how you're thinking about it.
Really fantastic work! Can't wait to play around with your library. I did a lot of work on this at a past job long ago and the state of JS tooling was so inadequate at the time we ended up building an in-house Scala visualization library to pre-render charts...
More directly relevant, I haven't looked at the D3 internals for a decade, but I wonder if it might be tractable to use your library as a GPU rendering engine. I guess the big question for the future of your project is whether you want to focus on the performance side of certain primitives or expand the library to encompass all the various types of charts/customization that users might want. Probably that would just be a different project entirely/a nightmare, but if feasible even for a subset of D3 you would get infinitely customizable charts "for free." https://github.com/d3/d3-shape might be a place to look.
In my past life, the most tedious aspect of building such a tool was how different graph standards and expectations are across different communities (data science, finance, economics, natural sciences, etc). Don't get me started about finance's love for double y-axis charts... You're probably familiar with it, but https://www.amazon.com/Grammar-Graphics-Statistics-Computing... is fantastic if you continue on your own path chart-wise and you're looking for inspiration.
Thanks - and great question about direction. My current thinking: Focus on performance-first primitives for the core library. The goal is "make fast charts easy" not "make every chart possible." There are already great libraries for infinite customization (D3, Observable Plot) - but they struggle at scale.
That said, the ECharts-style declarative API is intentionally designed to be "batteries included" for common cases. So it's a balance: the primitives are fast, but you get sensible defaults for the 80% use case without configuring everything. Double y-axis is a great example - that's on the roadmap because it's so common in finance and IoT dashboards. Same with annotations, reference lines, etc. Haven't read the Grammar of Graphics book but it's been on my list - I'll bump it up. And d3-shape is a great reference for the path generation patterns. Thanks for the pointers!
Question: What chart types or customization would be most valuable for your use cases?
Most of my use cases these days are for hobby projects, which I would bucket into the "data science"/"data journalism" category. I think this is the easiest audience to develop for, since people usually don't have any strict disciplinary norms apart from clean and sensible design. I mention double y-axes because in my own past library I stupidly assumed no sensible person would want such a chart -- only to have to rearchitect my rendering engine once I learned it was one of the most popular charts in finance.
That is, you're definitely developing the tool in a direction that I and I think most Hacker News readers will appreciate and it sounds like you're already thinking about some of the most common "extravagances" (annotations, reference lines, double y-axis etc). As OP mentioned, I think there's a big need for more performant client-side graph visualization libraries, but that's really a different project. Last I looked, you're still essentially stuck with graphviz prerendering for large enough graphs...
Ha - the double y-axis story is exactly why I want to get it right. Better to build it in properly than bolt it on later.
"Data science/data journalism" is a great way to frame the target audience. Clean defaults, sensible design, fast enough that the tool disappears and you just see the data.
And yeah, graphviz keeps coming up in this thread - clearly a gap in the ecosystem. Might be a future project, but want to nail the 2D charting story first and foremost.
Thanks for the thoughtful feedback - this is exactly the kind of input that shapes the roadmap.
