No, not at all. There is a transformer obsession that is quite possibly not supported by the actual facts (CNNs can still do just as well: https://arxiv.org/abs/2310.16764), and CNNs definitely remain preferable for smaller and more specialized tasks (e.g. computer vision on medical data).
If you also get into more robust and/or specialized tasks (e.g. rotation invariant computer vision models, graph neural networks, models working on point-cloud data, etc) then transformers are also not obviously the right choice at all (or even usable in the first place). So plenty of other useful architectures out there.
Using transformers does not mutually exclude other tools in the sleeve.
What about DINOv2 and DINOv3, 1B and 7B, vision transformer models? This paper [1] suggests significant improvements over traditional YOLO-based object detection.
Indeed, there are even multiple attempts to use both self-attention and convolutions in novel architectures, and there is evidence this works very well and may have significant advantages over pure vision transformer models [1-2].
IMO there is little reason to think transformers are (even today) the best architecture for any deep learning application. Perhaps if a mega-corp poured all their resources into some convolutional transformer architecture, you'd get something better than just the current vision transformer (ViT) models, but, since so much optimizations and work on the training of ViTs has been done, and since we clearly still haven't maxed out their capacity, it makes sense to stick with them at scale.
That being said, ViTs are still currently clearly the best if you want something trained on a near-entire-internet of image or video data.
Is there something I can read to get a better sense of what types of models are most suitable for which problems? All I hear about are transformers nowadays, but what are the types of problems for which transformers are the right architecture choice?
Just do some basic searches on e.g. Google Scholar for your task (e.g. "medical image segmentation", "point cloud segmentation", "graph neural networks", "timeseries classification", "forecasting") or task modification (e.g. "'rotation invariant' architecture") or whatever, sort by year, make sure to click on papers that have a large number of citations, and start reading. You will start to get a feel for domains or specific areas where transformers are and are not clearly the best models. Or just ask e.g. ChatGPT Thinking with search enabled about these kinds of things (and then verify the answer by going to the actual papers).
Also check HuggingFace and other model hubs and filter by task to see if any of these models are available in an easy-to-use format. But most research models will only be available on GitHub somewhere, and in general you are just deciding between a vision transformer and the latest convolutional model (usually a ConvNext vX for some X).
In practice, if you need to work with the kind of data that is found online, and don't have a highly specialized type of data or problem, then you do, today, almost always just want some pre-trained transformer.
But if you actually have to (pre)train a model from scratch on specialized data, in many cases you will not have enough data or resources to get the most out of a transformer, and often some kind of older / simpler convolutional model is going to give better performance at less cost. Sometimes in these cases you don't even want a deep-learner at all, and just classic ML or algorithms are far superior. A good example would be timeseries forecasting, where embarrassingly simple linear models blow overly-complicated and hugely expensive transformer models right out of the water (https://arxiv.org/abs/2205.13504).
Oh, right, and unless TabPFNv2 (https://www.nature.com/articles/s41586-024-08328-6) makes sense for your use-case, you are still better off using boosted decision trees (e.g. XGBoost, LightGBM, or CatBoost) for tabular data.
My best movie experiences were probably watching hard to acquire bootlegs in the pre-digital age. The barriers were just so much higher, half the excitement was just getting a crappy copy.
I don't know if it was on purpose or not, but I have heard it said more than once that Republican led states are able to greenlight projects faster, are more business-friendly environments, and generally have less red tape compared to Democrat led states. Love it or hate it, but greenlighting projects is a big component in allocating funds.
On the other hand, if they were so much better, shouldn't those "red states" be the ones with the much better economies?
I'm looking at China and what environmental price in the form of polluted land they pay and will be paying for a very long time. The big problems in Western countries also all originated in times with less or no regulation of such things. Just because that's not in the headlines all the time does not mean the problem is any less while the public is not paying attention.
When "it works" and overall success is the only criteria, the Vikings and Mongols surely count as extremely successful. Regulation is meant to take the price into account, in the cases of those two peoples millions of dead and a lot of pillaging and conquering. Regulation is definitely a burden, if you don't have to care about anything but the bottom line it's much easier.
Definitely not true. Right now Red states are openly attacking businesses that don't agree with the prevailing ideology. In Florida, the governor tried to destroy the state's biggest employer. In Texas they have been trying to prosecute out of state businesses. Alabama has more taxes on businesses than California.
I thought it is because they have worst economies and it is always an attempt to prop them up.
The other reason is that while republican party is purposefully trying to destroy economies of blue states, democrats were not trying to purposefully destroy economy of red states.
This might be true, it’s definitely repeated, but it’s generally not the real reason.
The real answer is just politics. Blue states have (generally) healthy economies, with a variety of economic actors and many businesses. Businesses often will shop around various states to build a factory looking for tax cuts. The politician can be associated with new jobs, and the business gets a discount, so it appears to be a win-win (if you ignore the lack of tax revenue). No one needs a tax break to start a business in NYC, LA, nor Silicon Valley, so you don’t hear about all the businesses that open there.
Nationally, policies like the IRA are big boosters for the economy, and democrats are focused on getting it done because it’s good for society and the national priorities. They won’t focus on where the money goes, and will allow the money to go to run down republican states as economic stimulus. But you’ll notice it’s usually capital intense factories that end up in these situations, not white collar jobs.
...thats how the US Constitution works. Congress passes laws (CHIPS Act) and the executive branch is empowered to carry them out - in this case the Secretary of Commerce and Commerce Dept. One can argue whether it stretches the intent of the law, nothing wrong with debate. But as of now, I don't think any judge or court has contested in the interpretation of the language.
The takeaway is the next Democrat president should just declare a public transit emergency and start building while the courts squabble. Same for housing reform. Same for climate change and shutting down coal power plants—once you shut it down and take out the turbines, it doesn’t matter what the courts say.
> as of now, I don't think any judge or court has contested in the interpretation of the language
Who has standing to sue here? The best I could see is a shareholder lawsuit, but that will take years. Meanwhile, this administration is getting slapped down by courts across the country, including a SCOTUS willing to overturn precedent to curry his favour.
Agree with this, but I think LLM's have been a net positive in helping generate commands? Admittedly, getting working commands is still tough sometimes, and i'm 50/50 on whether ChatGPT saved me time vs reading docs.
The other thing no one wants to say is the technical software/hardware talent in biotech is not as good as broader tech, and the people in charge of these projects are usually coming from a science background first and a technology background second.
A lot of the technology is regulated by FDA requirements, which apply to humans but not animals. Some of these requirements are challenging engineering problems - like how to do certain type of liquid transfers without exposing the contents of a container to the atmosphere, etc. There are lots of tools for doing liquid transfers that mimic what a human would do, but FDA places a different bar when it comes to developing a controlled manufacturing process.
There's also the question of, who writes the rules. Often it's industry experts that are already working on the process, and have incentives like keeping others out or licensing their own IP.
