My take wasn't that they are claiming SOTA, just that they are claiming same/better accuracy than whisper "tiny" and "base" models at higher performance versus original whisper.
The WER scores on the larger whisper models are lower (better) than "tiny"/"base", so none of these small models are close to SOTA either way.
I've played quite a lot with all the whisper models up to "medium" size. Mostly via faster-whisper as original OpenAI whisper only seem to care for/optimize performance for GPU.
I would agree that the "tiny" model has a clear drop off in accuracy, not good enough for anything real (even if transcribing your own speech, the error rate means too much editing needed). In my experience, accuracy can be more of a problem on shorter sentences because there is less context to help it.
I think for serious use (on GPU) it would be the "medium" or "large" models only. There is now a "large-turbo" model which is apparently faster than "medium" (on GPU) but more accurate than "medium" - haven't tried it yet.
On CPU for personal use (faster-whisper, CPU) I have found "base" is usable, "small" is good. On a laptop CPU though "small" is slow for real time. "Medium" is more accurate, though mostly just on punctuation, far too slow for CPU. Of course all models will get some uncommon surnames, place names wrong.
Since OpenAI have re-released the "large" models twice and now done a "large-turbo" I hope that they will re-release the smaller models too so that the smallest models become more useful.
These moonshine models are compared to original OpenAI whisper, but really I'd say they need to compare to faster-whisper: multiple projects are faster than original OpenAI whisper.
You are mixing the grid and retail electricity prices.
The retail electricity prices are linked to the wholesale gas cost. So consumers pay the same unit rate for electricity regardless of source/time of day etc. (simplification). Average 23p / kWh or so.
However, the producers of the electricity get paid the grid spot price. Plus any source-specific government-agreed subsidies (for nuclear, or the strike price for wind auctions).
So the typical/average UK grid electricity spot price is around £70 per MWh or 7p per kWh, so while retail is 23p / kWh a solar producer doesn't get paid 23p.
What you will also find is that the spot price, while it varies a lot, tends to be higher in the UK late afternoon / early evening, and higher in the winter than the summer, both of which are normally times when solar output is lower.
Some consumers, e.g. customers of Octopus energy, choose to pay half-hourly retail prices which shift as that auction spot price shifts.
Octopus isn't allowed to pass on the full burden (e.g. if the spot price soars to £5000 per MWh, which is crazy, Octopus has to eat that and charge their customers £1 per kWh as promised. even though that's much less than £5000 per MWh) and of course they want a profit (say spot price is £150 per MWh, Octopus charges 18p per kWh, they kept a few pence for operating costs and profit) but if you are able to react nimbly then this allows you to get a significantly better deal than if you're paying a fixed rate.
This is possible because of Smart Meters. The meter sends your usage, not once a month, or even once a day, but every half hour, so they can bill you 2p per kWh at 3am but 35p per kWh at 5pm reflecting the very different energy profiles and production.
Yes, I'm aware of the Agile Octopus tariff etc. but answered based on what >95% of UK retail consumers are doing.
From historical data, Octopus would also pass on a negative spot price to customers on that tariff so consumers could sometimes get paid to use electricity.
As you say I'd imagine that certain customers with high flexibility, maybe/particularly those with some battery storage (or an EV to charge), could make huge savings.
I don't know what sort of margin Octopus apply on that tariff, I expect it's a bit higher than the 3 p / kWh you suggest (when normal tariffs would average something like 15 p / kWh difference between average wholesale and retail).
Yes, I don't know what their margins are, including I don't know whether they recover their desired margins from all supply evenly (e.g. 15p per kWh all the time), proportionally (maybe 30p per kWh when wholesale electricity is expensive, 0p per kWh when it's very cheap) or according to some crazy formula, this strikes me as "secret sauce" for such a business.
For a huge fraction (maybe 95% for all I know) of UK retail consumers they're still with their legacy incumbent supplier, even though those deals are usually more expensive and the service is no better, "privatization" was largely a waste of everybody's time and money. But at least in principle they could all choose Octopus.
I take the opposite view - as more people have electric cars there are more batteries available. With some sort of dynamic pricing, and bearing in mind most commuter mileage needs only 1 full charge a week, you could encourage consumers to charge up when it's windy and/or hold off charging when it's not, so better matching demand to supply and reducing curtailment.
This is already a thing in some places. In Norway we have spot pricing of electricity, market price hour by hour. This means that it's possible to have a contract that lets you pay the market price at the time you use the electricity (timespotavtale). Even with the usual contract (spotprisavtale) where you pay the average price each day you can see that price (set by Nordpool) by looking on line or by subscribing to a service that notifies you when the price drops and decide whether or not to charge the car.
The Nordpool price today in the region where I live is 0.80 NOK/kWh, that's 0.074 GBP/kWh or 0.056 USD/kWh.
How many people actually do this and are happy with it though?
Here in the Netherlands there are also contracts like that, but people that are interested in them tend to put a lot of time and effort in then tracking those prices. I don't see regular consumers as a whole ever being interested in that. For now, the benefits are also small, and I happily pay a few percent extra (net) to not have to bother.
I also feel like it's not a good direction to move it: consumer energy markets are actually quite predictable so taking out long term contracts should stay the norm (consumer contracts and supplier contracts). Using car batteries as storage for solar can be addressed better by making it more attractive to charge at work, where I assume most park their EVs in the day, but right now pay full price despite providing a place to store surplus.
To me it made sense to price electricity at retail based on gas price when gas was the main source and the cost for that source is mostly about the market fuel cost. (There were and are other price guarantee schemes for nuclear where the ongoing fuel cost is only a small part of total costs. And the grid electricity price for wind is agreed by another set of rules - strike price).
However, we're now beyond that point so it seems to me that the pricing model should change. With the planned growth rate of new wind (and solar) by 2030 for the UK I assume it's going to happen one way or another.
23.4 p/kWh sounds about right. When prices go up next month (October 2024) I will be paying 30.5p / kWh day and 13.2p / kWh overnight (midnight to 7 am). Those are retail/consumer prices subject to government price cap, I believe commercial customers pay more.
Agree. It's about whether a buyer would reasonably expect the price to be correct.
UK law for consumer goods and sales, I would say generally doesn't require the buyer to be an expert. Laptops are items that are frequently included in sales and clearance sales at 50% off or more do happen. £400 is still a normal price to pay for a laptop. So I'd say the buyer could reasonably expect a laptop discounted to £400 to be the correct price, therefore HP were unreasonable to try to blame the buyer.
Of course. Small Claims process in the UK has a fee of about 10% of claim value and covers claims up to £10,000. If you win the other party has to repay those fees. No legal fees allowed. So would be the right process here - presumably as HP are in breach of contract the laptop buyer would win a case for a claim of the extra cost of the same laptop at full price.
That's the implication. If the distil models are same format as original openai models then the Distil models can be converted for faster-whisper use as per the conversion instructions on https://github.com/guillaumekln/faster-whisper/
So then we'll see whether we get the 6x model speedup on top of the stated 4x faster-whisper code speedup, at same/nearly same accuracy.
I would generally start with the assumption that if something is significantly faster the accuracy has to suffer a bit, but increasing model size and/or settings such as beam size to compensate should allow same accuracy and higher performance (just not all of the stated performance gain).
Yes, it's not that clear to me either what test sets get a 10% error rate. Because in my use (native English dictation or native English podcast transcription) the small or medium original whisper models have what I'll call a "discrepancy" rate of say 1-2% which is mostly punctuation and "umms/errs" inclusion or not. The actual "error" rate is below 1% in my experience, and excluding surnames, brands and place names that I don't know how to spell either the remaining errors tend to be minor (missed plural etc.).
So I infer that these data sets are some deliberately difficult audio: call centre recordings with lots of background noise, phoneline quality audio etc. Maybe non-native speakers. If I only heard that sort of audio once I also might have an error rate of 10%.
The WER scores on the larger whisper models are lower (better) than "tiny"/"base", so none of these small models are close to SOTA either way.