Sure, there are many ways one can be accidentally inefficient, some examples:
Batching: Your throughput (and thus cost per token) will be much worse if you don't do batching (meaning running inference for several inputs at once). To do batching, you either need to have a high sustained QPS or adjust your workload to be bursty (but this has utilization issues).
Inference engine: The regular HuggingFace, even with built in optimizations, is not competitive against inference engines like vllm, exllama, etc.
Utilization: Depending on your scale, it might be hard for you to have a nice flat utilization, or to even utilize one GPU at 100% capacity. This means you need to solve scaling up & down your machines and the issues associated with it (can you live with cold starts? etc.)
Hardware: Forget server-less GPUs like Replicate etc, their markups compared to on-demand pricing is usually >10x. On-demand A100 or H100, provided you can even get a quota for them, are also expensive. Spot instances are better. Older GPUs (A10, T4, etc.) are better. Your own GPU cluster is likely the best if you have the scale (and likely, you can resell your cluster within the next 18months without much if any depreciation).
For these reasons, I have been toying with the idea of providing a dead simple service for fine-tuning & serving open-source LLMs where the users actually own (and can download) the weights. If anyone is interested in this and would like to chat, let me know.
L&K, are two PHDs taught by their now dead former professor. Their research into LK-99 is inherited from their former professor but they lacked funding, hence it was a hobby project for a long time, and they needed funding to work on it full time.
The Korean professor: A professor who started taking interest and sponsoring the project for the last few years. However, he likely didn't participate directly in the research much.
The American professor (Ethnically Korean): A professor who was brought in rather recently (Maybe last 1-2 years?), to help bring credibility to the team. He is probably the one with the expertise to perform measurement and ascertain that it is a superconductor.
The Korean professor got kicked out from the team for unknown reasons. This was evidenced in the April paper published in Korea, which lacked the Korean professor's name. Quantum center also removed him from the website.
4 months later, he was back with a vengeance, and published the paper, with only him, and L&K on it (ie, the 3 man paper). The Korean professor possesses no sample, and also lacks the original data for LK-99.
A day later, the american professor hastily rushed out another paper (The 6-man paper), with L&K, himself, and 3 irrelevant lab assistants.
Since the earlier Korean paper in April got 0 attention and was treated like a typical troll paper. They probably assumed posting on arxiv was just good material for later legal battles, but they could otherwise continue happily researching and patenting their material.
However, interest in LK-99 exploded like a nuke, because the first signal for credibility: That two professors were in a bitter battle over authorship, did not escape people's eyes.
2 days later, the Korean professor had a presentation at a Korean conference, but attendees were disappointed by his data and lack of samples (Prediction markets hit a low of 15% at this point).
However, the American professor took control of media communications, and quickly dissociated with the Korean professor, yet he completely stood by the claims of LK-99. Now the American professor has talked to numerous Korean media and directly sent an exclusive video to NYT.
Its interesting why L&K chose the American professor over the Korean one. Needless to say, if this works out, the interpersonal saga will end up in a Netflix documentary, and probably many court cases.
I should also note that the nobel prize is likely a secondary concern for the group. They aren't in a comfortable academic position where money is only a bonus. They were struggling PHDs for a long time. Nobel prize money is too little, and will come way too late to make a difference for their lives.
L&K and the American professor are likely trying to maximize the value of their patents and know how to sell to investors, rather than to make public verification ASAP, which would be optimizing for a nobel prize.
There are two things that are referred to as "fuzzy logic".
The first, is the true fuzzy logic, which consists of Fuzzy Set theory and Fuzzy Measure theory. The two are distinct but interrelated, and heavily mathematical in nature. In particular, Fuzzy Measure theory completely subsumes Probability theory (i.e. probability theory is a strict subset of fuzzy measure theory, and therefore anything that can be expressed in probabilistic terms can be expressed in equivalent fuzzy measure theory terms, with leeway for added generality).
The second, is a class of techniques, based on Fuzzy Set / Fuzzy Measure theory, which implement some very simple behaviours which allow you to solve a particular class of problems. This is the standard Sugeno/Mamdani logic, fuzzy inference as the application of a few straightforward rules for conducting conjunction/disjunction among fuzzy rules.
The first is still thriving, but largely in the theoretical realm.
The second is shunned, on the basis that it feels like a bunch of interpolation heuristics, and a Bayesian approach is usually "better", even if not as intuitive of straightforward.
Unfortunately this has had a self-prophetic impact, in the sense that any literature which would have been better described as fuzzy, is labelled as probabilistic, even if strictly speaking the concepts involved are not strictly-speaking probabilities, but should more correctly have been addressed as fuzzy measures.
Personally I have published papers where I had to "disguise" fuzziness as probability just to get it considered for publication, or where I had to "apologize" for the term, or making it explicit that fuzziness in this context is unrelated to this set of tools that is typically thought of as "the fuzzy approach".
Having said that, one area that is thriving is the use of Fuzzy methods in the context of more general AI / NeuralNetwork methods, particularly in the context of explainability. Type 2 Fuzzy methods in particular (which introduce one or more layers/dimensions of fuzziness/uncertainty over the membership function itself) are quite an active area of research.
Finally, there are some frameworks that borrow heavily from fuzzy theory, but strictly speaking are independent. The one that interested me the most in recent years was "Subjective Logic", proposed by a guy called Audun Jøsang; this sounded like a very interesting logic framework and I was very keen to use it in my work, but in the end I just didn't have the time to justify that investment ... It's worth looking up though. The basic idea is that you have a framework where you have rules that have a built-in way of carrying uncertainty with them, with a one-to-one mapping with beta probabilities over your fuzzy-like rules, and a set of logical operations (which map to the usual logical operators) that are adapted to carry the uncertainty with them. This means that you can reduce your problem into a set of simple logical statements, and the framework would then take care of the uncertainty calculations for you seamlessly.
People really need to update their model of what a "statistical predictor" can accomplish.
We know that Transformers are universal approximators of sequence-to-sequence functions[1], and so any structure that can be encoded into a sequence-to-sequence map can be modeled by Transformer layers. It follows that prediction and modeling are not categorically distinct capacities in LLMs, but exist on a continuum. How well the model predicts in a given instance is largely due to the availability of data during training. This is the basis for the beginnings of genuine understanding in LLMs. I talk about this in some length here[2]. Odd failures and hallucinations are just the model responding from different points along the prediction-modeling spectrum.
The pre-trained model is stage 1 - it has seen everything, but it is wild. If you ask it "What is the capital of US?" it will reply "What is the capital of Canada?"...
Stage 2 is task solving practice. We use 1000-2000 supervised datasets, formatted as prompt-input-output texts. They could be anything: translation, sentiment classification, question answering, etc. We also include prompt-code pairs. This teaches the model to solve tasks (it "hires" this ability from the model). Apparently training on code is essential, without it the model doesn't develop reasoning abilities.
But still the model is not well behaved, it doesn't answer in a way we like. So in stage 3 it goes to human preference tuning (RLHF). This is based on human preferences between pairs of LLM answers. After RLHF it learns to behave and to abstain from certain topics.
You need stage 1 for general knowledge, stage 2 for learning to execute prompts, stage 3 to make it behave.
I cringe to think of myself as a "lifer" but I've been doing this for 10 years so draw your own conclusions.
Above all, the thing that makes me stay is having creative control and freedom over how I do my job. I've done enough things to know how rare that is, and how frustrating it is not to have it, and what a blessing it is to have the trust of the people I work for and with. It's so important to me that I feel it as a different baseline state in my body.
I'm currently going through a youtube (edx?) video course[1] based on htdp, which I found through this hackernews comment[2].
I like it so far, but was initially surprised that there's a systematic way of designing programs. I've just been winging it throughout my self-taught career.
It's worth it for me in the sense that I blog to process my thoughts, learnings, projects, and experiences. Blogging helps me do that, and since that's the point, it's worth it.
In terms of getting noticed/finding jobs, my blog (https://liza.io) is not super high traffic or anything. I don't market it. I don't even usually tweet about my new posts, even though Twitter is my one active social media platform. But I have had recruiters mention checking out my blog posts and sometimes reference my blog content. I've also had interviewers mention my blog and ask about personal projects I've written about in the past. I also used some of my blog articles as supporting material when applying for a position that featured a lot of technical writing. So I think that even though improving job prospects is not the primary goal of my personal blog, it has been a bonus.
Another bonus has been getting occasional emails and comments from random readers to say a certain post has helped them. A while back I wrote a post about testing external API calls in Go, and have had a few people reach out to thank me. Though I write primarily for myself, it's always nice to see when someone stumbles across my content and appreciates it.
It's also always cool when a post ends up in newsletters I've never heard of. I don't have a consistent way to detect each time this happens, but can sometimes trace back traffic through a referrer or notice a backlink from a newsletter archive.
Real-time by default for everything is really a fun way to write web apps. A little bit inspired by Meteor we've build Thin Backend, which provides a real-time API for querying data and writing to a postgres database. If you're interested, check it out at https://thin.dev/ or check the demo video https://www.youtube.com/watch?v=-jj19fpkd2c
Having had the pleasure in training three kids on how to ride a bike, forget training wheels, or taking the pedals off!
find a reasonable long grass hill - not to too steep, but steep enough to allow a kid to go down the hill riding the bike with out pedalling.
and then goto the top of the hill and let them go down the hill on the bike. Once they master the balancing part of going down, they seem to pick up the art of turning the pedals at the end of the hill to keep going (if not, explain what to do)
advantages: doesn't hurt when they fall off, gets them used to rough terrain. it becomes fun once they start to master the balancing bit
The US government (hence the people of the US) repay their debts all the time, and issue new debts all the time. Just like a home mortgage gets sold from the original lender to subsequent lenders, the borrower still usually pays back the mortgage.
If the debts were not paid back, then why would anyone want to buy the debts from the original lender?
Isn't what you're describing a symptom of the natural and inevitable crunch into the singularity of digital indoorism?
Going outside at all is a hangover of a rapidly departing age. There's nothing out there anymore. Everything's in your home, except for food. Have it delivered, and you're solid. That just leaves commutes. Insofar as they're necessary, I agree with living in dense, hyperurban hive arcologies where everyone can walk to work/school.
Outside is gauche. It's jejune. Meanwhile, commuting's a necessary evil, and cars are one of the most dangerous and inefficient ways to do it. Update to the new meta or farm outmoded vanity betrayed by its sprawling, parking lot-shaped contradictions.
Humans have a larger neocortex compared to other mammals, which gives them more storage, working memory, and computational potential than other animals.
The curses of the human condition are anxiety and regret, the conditions that render people unable to engage in the present moment. The former is characterised as being stuck in the future psychologically, and the latter is being stuck in the past. Since neither the future or the past can be changed by obsessive thoughts in the present, these psychological conditions are a recipe for unhappiness, because happiness can only ever happen in the present moment, which is lost.
What characterises flow state and mindfulness is total absorption in the present moment.
What if animals that are not burdened by human quantities of neocortex, such as insects and canines, which exist in the present moment, are actually experiencing flow and mindfulness?
The spider weaves its web in perfect concentration, not stopping to contemplate, plan, or exercise executive judgement. Might not that be akin to flow?
The dog and the cat and the cow and the horse, given physical comfort, do not seem to worry about the future or regret the past. Might they not be blessed with a form of mindfulness that humans in the rat race rarely manage to experience?
Sure it would be nicer if good products were automatically successful on their merits alone. But the world is a very, very big place. Therefore a major part of the problem resting on your shoulders as a business owner is getting your product into the attention spheres of people who will pay for it.
IMO it's better not to fight realities you can't change. The best you can do — from both a business and a mental standpoint — is accept that marketing is a necessity and find a way to enjoy it.
I'm a programmer who's been indie for 10 years and I treat marketing as a series of open-ended experiments; this arouses my intellectual curiosity. I also gamify it by gathering real-time stats; this emulates the dopamine hit I get when a piece of code compiles.
I've vlogged and blogged a good bit about marketing from a programmer perspective, so if you'd like more on my take, check the links in my profile.
I've spent a ton of time as a developer trying to make money from various side projects and businesses. So most of my top "wish I'd discovered this earlier" list revolves around tech+business stuff:
* Strategy #1: Charge more. patio11 has been shouting this from the rooftops for years, but it didn't sink in until after I started Indie Hackers[0]. If you charge something like $300/customer instead of $5/customer, you can get to profitability with something like 50 phone calls rather than years of slogging. It's still hard, but it's way faster.
* Strategy #2: Brian Balfour's four fits model[1]. It's not enough to think about the product. You also need to think about the market, distribution channels, and pricing, and how each of these four things fit together. I imagine them as four wheels on a car. It's better to have 4 mediocre wheels than 3 great ones and a flat.
* Book: The Mom Test.[2] Amazing book about how to talk to customers to research your ideas without being misled, which is a step I've stumbled on before.
* Tool: Notion. I just discovered it recently. I use it for all my docs and planning.
TL;DR: there are disadvantages, technically, but they're irrelevant compared to the advantages. It's true that human contact is a physical need [and that is physically perceptible, if you're the perceptive kind], but work, unfortunately, is definitely not the kind of place for physical contact. Instead of socializing at work, now I see my family on a daily basis by having more time to go out and visit them daily. (Being a grad student, I also enjoy a little physical contact with people at the university on a weekly basis.)
I've been working remote 100% of my time for about 3 years, spanning two different companies. The first year was with a company in my city, people I never knew before. I was hired by them to do some easy work and they proposed me some harder work one week after watching me solving the initial problems. I solved that too and little by little impressed them with competence. So, professionally, things were in perfect shape, but, on a personal level, things were going from bad to worse. So after delivering one last promise, I told them I was starting my own company and I could continue delivering them more services --- for instance, in my own office.
So I got to work remote. Initially I saw them once a week, then a few months later, once every two weeks and then once a month. After they got one hard project completely done, they didn't have anything they didn't know how to do, so they didn't ask me for anything else and I had to find a new deal.
Despite my very polite and elegant way of dealing with them (as I do with everyone), despite getting lots of compliments from these people due to the quality of work and so on, no one would ever convince me these people were not crazy to see me away. They only tolerated me while they had to.
So I called a company I worked with for many years. I closed a deal on that very call because I made them an irresistible proposal. I'm very far away from them, so the work must be 100% remote. So it's been like this for 2 years now.
Despite being remote, we meet through cameras every day. (They do all the SCRUM nonsense, JIRA and so on, though I'm the only programmer they have. What they see in these tools is a fancy post-it that gives them a feel they're doing some management. So, yes, I've no respect for these tools. I'm definitely the more organized person in the team and all I use is GNU EMACS and org-mode. I'm definitely not suggesting they should use such tools. They're not programmers. I'm just explaining what is the environment of such offices that I'd have to deal with if I were there in person.)
I clearly respect their ways of working and fully collaborate. I'm probably the easiest person. Everything they ask me, no matter how nonsensical it is, I always agree and proceed to get it done. This is very easy for me to do precisely because I'm remote. All the lies and superficiality that goes in an office is dealt with even with humor if you're remote. When I see someone acting totally out of harmony, I can even joke around aloud so as to not feel too downhearted about it. Nonsense on the screen is much less frustrating than when you're on the spot. You can watch a bad film on TV and turn it off. The same happens working remotely. The nonsense goes on, you respect people's ways and as soon as it's over, you close the camera and back to your coding, to your work, which you do with excellence, not because of talent, but because you carefully check your calculations, you know how to add two and two and you're never a fool to think you don't have to check the basics. That is admirable, that makes the day worth it. So, working remote you live in your environment which is made up of the things you do. The inner is the outer: when you're alone, the things you have inner are the things that fill up the outer.
If by being alone you get down, don't eat well, don't catch sun, then that is the fact, that is your life and you should not panic, but carefully watch the problem and let it tell its story to you --- which is the story of your life. I do not think intellectuality is what is needed to live a good life, but there is not much difference between learning to live well and learning technical things. Watch it, watch the problem, let it tell its story, be humble, check the basics, learn with those which factually know more than you.
I've known people who had remote jobs and due to the difficulties of being alone decided to go back to punching cards in standard offices. That is fixing a bug by causing another. Do not do that. Look in the eye of the storm. Do not pretend to solve problems. I wouldn't mind spending my whole life trying to only understand the Pythagorean theorem: it is not higher mathematics that matters; it is the learning.
Work is central to life. You should own your work. After you've invested decades in a company, that company is yours, and as such it should be impossible for people to take you away from your work, which is absolutely one of the major parts of your life. So work should be the place where you feel great, where you can be so free and be able to watch your development as a complete person, not as a machine with a function and a schedule.
You might think this might all apply to me because I'm a highly unusual person. But that would be a total miscalculation. I'm convinced everything I say applies to everyone else. I consider the most typical person there is out there and how they could never work alone in their own workplace because they'd get down, be lonely, nervous and so on. So such person feels working in an office or in a factory is a marvelous thing for them. But the fact is that they only think that because they're deeply afraid of facing their real problems, which is buried in themselves. Nowadays every one is working on a computer terminal, so there is little reason to cruise the city to punch cards in an office.
In the last year I've learned about the worth of a man, how big a burden a family can bear, and the extraordinary lengths a mother will go to save her dying child.
Last July, I was fired from my dev job. This was a shock to me as I had been seen as a high performer at my last job. Well, my last job before my last job at least. I was scared, confused, and really down on myself after this happened. I'd quit the job before because it seemed like a toxic environment. My wife was pregnant, I applied for unemployment which was embarrassing and felt pretty bad. Honestly the place was pretty bad but I tried to hold on because of my wife's pregnancy and I knew it would look bad to lose/quit two jobs back to back like that. In retrospect I would just leave randomly at 3pm, "work from home" (take naps), and was overall a pretty terrible worker during my tenure there. After losing that job, I didn't just "bounce back". I got depressed. I thought about what was different compared to my jobs before, and came to the obvious conclusion it's the adderall, stupid. Taking adderall turns me into a super performer. It makes me really, really good. It also makes me feel like I'm "broken" in some way that has to be fixed by a drug. I'm simply not a great developer and can't achieve the intellectual goals I have for myself without taking it paired with a mild anti-anxiety medication. Eventually I decided to start taking it again after a year and a half without it. I just got my performance review at work and got glowing praise from my manager. It makes me feel good. It makes me get rewarded financially. It's also interesting because there's two groups of developers that have very, very different opinions about who I am and what I'm like to work with. The first class has worked with me drug-free. These people don't much care for me. I'm forgetful, get distracted easily, will constantly be away from my desk walking around, if I'm even in the office. The other group have worked with me and see me as someone who can solve any problem they throw at me. I'm getting the reputation as the one to give complex and difficult problems to. People think that other people are immutable and can't change, too. Both groups have fixed about me. If I stopped adderall tomorrow my coworkers give me the benefit of the doubt, feel that if I'm stuck on a problem for a long time it must be because it's complicated, not because I'm astutely avoiding solving the problem. The other group just thought I was a total loser. I hated it, because they were right. I know what I'm worth, and what makes me worth that. It's not all me.
Last October, bleary-eyed and exhausted at five in the morning, my wife gave birth to a beautiful baby girl via c-section. The labor had not gone well. Her water had broke and there was meconium (baby feces) in the water, which is a sign that the baby is in distress. We rushed to the hospital and my wife labored for eighteen hours, screaming "just cut her out of me, something is WRONG!!!". The doctor ignored her and encouraged her to keep pushing. Finally, he relented and a Cesarean section was performed. The nurses took her away. Someone told us "Your daughter has myelomeningocele. Listen to me. Don't Google it. Just get some sleep.". So I did. I remember waking up to the children's hospital ambulance crew asking my wife to sign the documents to transport Lucy to the hospital. She had been born on the same day as her namesake, her great-great-grandmother. We didn't have Thanksgiving last year. We didn't have Christmas last year. There was only the endless march of days at the hospital. I accepted my new job the day my wife went into labor. I just had to deal with it. What else could I do? Throwing myself into the work certainly helped me forget that my daughter might never walk, that she probably won't be able to control her bowels, that she'll need to catheter herself whens she's older to prevent her kidneys giving out. At a happy hour, I mentioned to some coworkers about her condition. A gray-beard developer took me into a back room and got really serious. For a second I thought he was going to fire me, honestly. He said "I want you to know I have spina bifida. Everything is going to be okay." That's one of the nicest things anyone has ever said to me.
In January, they told us Lucy was going to die. There's a malformation of her brain stem, the part that controls unconscious breathing. When she got upset, she held her breath. It's a horrific thing to see, your child turn blue. Then they turn white. She looks dead. As a gentle snow fell outside, my wife performed CPR on our four month old while I frantically called 911 from my wife's phone because mine had just died. I'm a dumbass. Our four year old is twirling in circles as the paramedics arrive. She asks the cop if he can open her milk, she's thirsty. Lucy had started breathing again. We told the paramedics the doctors had said this might happen, but that we should call the paramedics "just in case she doesn't recover on her own". The paramedic looks at me and asks "what's her prognosis"? I didn't really understand, but she was asking "is your child going to die?". They took her and my wife away to the hospital. The doctors read some research and told my wife that the majority of kids with this complication, called PEAC, die. They don't ever "get over it", even kids as old as 12 have died. We ask if there's anything we can do. The doctors shrug and the palliative care psychologist wants to talk to us. My wife spends the next few days glued to the computer. She asked the doctor for the print outs of the medical research he's referencing. She finds more. She finds a doctor at the Mayo clinic who had done some research prescribing a specific anti-anxiety medicine and found that the mortality of these children had gone to zero. Admittedly it was a small sample size but initially the doctor brushed off the suggestion of the medicine. My wife wrote a long email with citations describing Lucy's symptoms, and the treatments outlined in the medical literature. She emailed it to the hospital's social worker since you can't communicate directly with the doctors. The next day the doctors had the bright idea of prescribing the anti-anxiety medication. Lucy's breath holding spells stopped. There's a surgery that's also suggested in the literature. My wife asks for that. The head of neurosurgery says he won't do it. We call a doctor who is an expert in her condition and he says she should get the surgery. We say we want to transfer to that hospital, hundreds of miles away. Once he hears that, the head of neurosurgery says he'll do it. My wife sighs in both exasperation and relief. The doctors have fought us every step of the way.
Lucy came home two weeks ago. She's actually a pretty normal baby. Cognitively she's right on track, even though she doesn't wiggle her toes. We're provided with a night nurse twelve hours a day because her breathing can be irregular sometime. But her breath holding spells, the thing that was most likely to kill her, have completely stopped. Things are starting to feel normal, the nurses are kind to us and to our daughter. My wife seems to be relaxing for the first time in eight months. It looks like she has a future. Life is no longer paused until further notice.
I've learned I'm worthwhile. I'm more worthwhile if I take stimulant drugs. I've learned that a family can endure far more than you might think it can. I've learned that my wife will move mountains if she has to for our kids.
As an experiment, I started https://wordsandbuttons.online/ in plain hand-crafted HTML+CSS+JS (JS for the interactive bits only). Every page is self-sufficient, there are no dependencies, no dynamic data exchange, no HTML generators, - nothing. And it works so well, I don't really want any frameworks now.
It's perfectly malleable - I can make every page do exactly what I want it to. It's lean and fast - every page even with all the interactive plots, games and quizzes, and syntax highlighting, is still under 64K. And it doesn't accumulate technical debt. There are no dependencies, even within the project. An occasional bug from the past wouldn't bite me in the future.
At the beginning, I thought that typing the HTML by hand would be too tedious. But it's not. It is a bit boring, sure, but it's only like 5% of all the work so usually I don't really want to automate it.
And when I do, I just write scripts that handle HTML just like any other text. Works wonders for me.
Deadlines are effectively reality for many projects because tons of people and scares resources need to planned around them. Rallying against them is unrealistic
Let me give some examples:
You're making a product for retail. The stores that will carry your product need to plan to have their shelves full of product. That means they need to know when your product will be available. What promotions will appear on posters, flags, etc in the store. They need the plan this 6 months in advance so you have to commit to a deadline well beyond your finish date.
You plan to run commercials. The TV companies need to know sell their commercials months in advance. So you commit $$$$$$$ to pay for those commercials 6 months in advance. Miss your deadlines and there will be no product when your commercials run.
You're adding features to a mobile OS. Your partners (Samsung, or Sony, or LG or Motorola) are planning all of the stuff above (PR, Commercials, retail placement). They are going to be touting your new feature. Because they have to commit to ads and shelf space months in advance you also had to promise your feature will be ready months in advance.
I'm sure there are better examples. Here in Tokyo they getting ready for the Olympics. The venues, transportation changes, etc have a hard deadline.
Video games used to (still?) have all the same issues. 6 - 9 months before Christmas you have to promise the stores your game will be ready and on their shelves. That may change in the future but even in 2019 retails sales of bluray PS4/Xbox and Switch SD games are greater than online for the majority of games.
If you happen to be on a project that can run without deadlines and things get done and shipped whenever then lucky you.
After many decades of coming up with ideas, I have discovered something. Every single one of my good ideas have been ideas that somebody else also had. It shouldn't be surprising because there are something like 7.7 billion people in the world. If you have a good idea that nobody else had, then that's a 1 in 7.7 billion idea ;-) Having realised this, I now use the fact that the idea is already known to be a good test of how good the idea is. If the idea has already been tried then I get an even better idea of how good it was.
Here's the thing: the world is not short on good ideas. Actually, you don't even need to have one to be successful. You just need to be able to recognise a good idea. The rest is execution. You execute better, you will win bigger.
Here's another cool thing I learned in decades of looking at the technology world: name one dominant company whose product was the first to market in its field. I literally can't find one! They probably exist (and this is HN, so there will be someone coming along shortly to point it out to me ;-), but the most important thing to realise is that most (by an incredible margin) successful projects became successful be learning from the previous king in its area. Word processor, spreadsheet, OS, game (of any genre), source control service, etc, etc, etc.
Don't be afraid not to innovate. Learn how to deliver. Learn how to distinguish between good and bad. Learn how to optimise the good. Learn how to please unhappy customers of existing products. Learn how to do all that while making money, not spending it.
Or the other way is just to close your eyes and pretend you invented it. If you become dominant, nobody will realise you weren't the first ;-)
It's noteworthy that as you have shown, functional languages still have this insane opposition in the industry because they are perceived as toy languages.
Despite the fact that they have a very bullish minority in the industry doing some very serious and lucrative work using them.
See for example Jane Street and Ocaml.
The fact that functional languages are not yet "mainstream" is more of an artefact to two things unrelated to their capability as languages in an industrial setting:
a) Historical baggage. Industry has always taken the first tool it has come up with, declared it the "professional tool", which has become a self fullfilling prophecy then. Despite when looked from historical perspective, there is no technical reason why the chosen language was best among many. See for example C, C++, Java, and my favorite example JavaScript (ship it! 9 days is quite enough to make a language).
I.e. industry has taken anything that creates an abstract syntax three that can be transferred to the chosen execution context, and maked it work by adding tooling and education, no matter how crude the language itself was.
b) Lack of industrial quality tooling. See a). Industrial quality tooling is often a sane prequirement to choose a language for a project. There might be instances where you feel you might want to hire a legion of developers, and then you choose probably one of the most popular languages because it has tooling and it has a large number of candidates who know the language.
Note: neither the fact that the language has tooling, nor that is popular, does mean it's the "best" for an arbitrary scenario.
Batching: Your throughput (and thus cost per token) will be much worse if you don't do batching (meaning running inference for several inputs at once). To do batching, you either need to have a high sustained QPS or adjust your workload to be bursty (but this has utilization issues).
Inference engine: The regular HuggingFace, even with built in optimizations, is not competitive against inference engines like vllm, exllama, etc.
Utilization: Depending on your scale, it might be hard for you to have a nice flat utilization, or to even utilize one GPU at 100% capacity. This means you need to solve scaling up & down your machines and the issues associated with it (can you live with cold starts? etc.)
Hardware: Forget server-less GPUs like Replicate etc, their markups compared to on-demand pricing is usually >10x. On-demand A100 or H100, provided you can even get a quota for them, are also expensive. Spot instances are better. Older GPUs (A10, T4, etc.) are better. Your own GPU cluster is likely the best if you have the scale (and likely, you can resell your cluster within the next 18months without much if any depreciation).
For these reasons, I have been toying with the idea of providing a dead simple service for fine-tuning & serving open-source LLMs where the users actually own (and can download) the weights. If anyone is interested in this and would like to chat, let me know.