And that’s fine. Another follows after. If we leave them something.

Part of the problem is that industrialization was achieved by exploiting globs of easily available resources. But we used them all.

We haven't left anyone something. It could very well be that we climbed the ladder and burned it behind us.

This is part of the civilizational collapse narrative. It is definitely true in a way.

I think that how much it would end up mattering depends on how well solar tech would withstand a civilizational collapse.

I think that a proto-industrial society with photovoltaics and batteries would be able to bootstrap itself back up to the present state, even without easily exploitable fossil fuels.

(I am not an expert in any of this)

How are they going to make pv’s and batteries in the first place?

Biomass can substitute. You have to defend it, which is difficult if rivals have petroleum. But that wouldn’t be an issue if nobody had petroleum.

We haven’t burned the ladder.

What do you mean, we haven't left anyone something?

I've heard a few people say we've already drilled all of the easy-to-get oil, so the next civilization may not develop using oil.

But maybe all that means is they will master some other technology that we did not. It seems like previous civilizations have mastered technology that we cannot figure out.

> It seems like previous civilizations have mastered technology that we cannot figure out.

That was true for a while (e.g. in Britain they spent 500 years figuring out what the colonising Romans had done to try and recreate their advances), but I don't think we're now in that situation. The best thing would've been to mostly convert electricity to nuclear in the 1980s, as by now we'd be handing over ultra advanced nuclear designs and low carbon to the future. Now we're just playing catch-up with the 1980s.

Colonialism never really ended it just transitioned into a different form, sometimes even very overtly like parts of africa are still using the french colonial currency union (CFA) for example, the IMF keeps the global south in debt entrapment with structural adjustment programs designed to prevent development. etc. etc. we never really left them alone

> IMF keeps the global south in debt entrapment with structural adjustment programs designed to prevent development. etc. etc. we never really left them alone

Countries invite IMF assistance. If they wanted to be left alone, all they have to do is do nothing. If IMF loans didn't have strings attached, they wouldn't be able to borrow money, as it's those strings which build bond investor confidence. The entire point of IMF assistance is to avoid being cutoff from international borrowing for being horrible credit risks (again).

The root cause of national debt problems is primarily government corruption, but also mismanagement, often at the behest of populist politics that excuse economic policy failures by, e.g., scapegoating outside forces. The US isn't immune to this problem, either, it just happens that the US had, albeit intermittently, long enough runs of solid financial management (e.g. Hamilton during the Founding) that it could grow an economic base that could withstand intermittent periods of mismanagement without the entire economy collapsing (yet).

Even when a country is dealt a really crappy hand at the outset, it's not irreversible. Haiti is the poster child for crushing debt unfairly imposed by foreign powers, yet the Dominican Republic had the same history, but managed to overcome it. In some instances, interventions blamed for keeping Haiti oppressed were precisely what helped the Dominican Republic flourish. Likewise, nobody hears about the IMF success stories, just the failures; and it's not because the former don't exist or are rare.

> Even when a country is dealt a really crappy hand at the outset, it's not irreversible

Hong Kong was poor until 1965, when they got tired of poverty and switched to free markets. The result was amazing prosperity.

> switched to free markets

Hong Kong has been all about free markets since the end of the Opium Wars.

> government corruption > mismanagement > populist politics > interventions blamed for keeping Haiti oppressed were precisely what helped the Dominican Republic flourish

The rhetoric transitioned into exactly this, instead of believing they were subhuman uncivilized people we needed to save from themselves (the white mans burden), it seamlessly transitioned into neoliberal ideas of sound economic theory seeking a "scientific" rationalization of why those neoliberal policies forced onto them fail them consistently and how it's actually all their fault. Any sovereignty is reframed into dangerous intolerable "populism" that needs to be crushed by any means necessary, including crushing sanctions and blockades (stop hitting yourself), covert actions, coups and military interventions.

I certainly didn't assign moral fault to anyone or any group. Indeed, framing it as a moral problem is, IMO, one of the problems here. A country isn't run like a business; similarly, collective morality doesn't look anything like individual morality, assuming it's even a thing at all.

Corruption being a root cause for impoverishment is a fact. How corruption arises, and how to get out of that local equilibrium, is a difficult collective action problem without any easy answers, though there's countless books on political and economic development that explore it. Colonial oppression is a horrible explanation as it has very poor predictive power, unless you define colonialism in a conclusory, tautological way; and even then, it does zilch in terms of identifying effective solutions. Indeed, relying on an oppression narrative is one of the ways corrupt governments and elites justify and excuse the consequences of their policies.

That said, "corruption" isn't a great explanation, either, but it's certainly better than the colonialism morality narrative. Unless someone has lived in some of these poorer countries and witnessed the extremes of corruption, they tend to equivocate all kinds of corruption, and when from wealthier, more democratic countries are unable to distinguish or even imagine what severe, pervasive corruption looks like and how it effects every aspect of society.

> A country isn't run like a business

I'm not convinced you really mean that, but I agree they shouldn't. Although we've invaded countries that tried that (and are in the process of invading a few more while we are speaking).

> though there's countless books on political and economic development that explore it

we clearly have read very different books on the matter. What is the answer to corruption given by neoliberalism? Isn't the very policies enforced and implemented in the global south believed to combat corruption? Hasn't that demonstrably failed them? But people like me take issue with the whole corruption narrative, we would argue the west, especially the US is the most corrupt nation on the planet by scale, we just don't call that corruption, we just give it names like "lobbying" or "stock buy backs" and make it legal.

> Colonial oppression is a horrible explanation as it has very poor predictive power

You can see colonialism from space, with old rail lines and other infrastructure leading from the mines to the coastal cities, it literally shaped their geography, their colonial history is the single most important unimaginable violent event that has ever happened to these nations, its inseparable, it shapes their past, present and future. It has absolutely predictive power, it shaped them and our grasp on them to this very day is undeniable reality for those nations.

> unless you define colonialism in a conclusory, tautological way

We absolutely have to study colonialism as a distinct, special thing, we need to understand how this legacy shaped them and our(western) relationship to them to this day. We didn't just pack our bags and left them alone. Everyone recognizes that, it's not like we don't care, we do all kinds of things in development, its just we should observe why this all made so little progress despite 75 years, billions in aid and one failed IDF program after the other.

> relying on an oppression narrative is one of the ways corrupt governments and elites justify and excuse the consequences of their policies

you could say the same about the corruption narrative, it ignores things like effects of globalism and military interventionism too, and has served our own elites VERY well.

> That said, "corruption" isn't a great explanation, either ...

I don't get the point in your last paragraph

Interesting. 25000 hours actually.

Having to pay for utilities you mean?

Gains from efficiency are experienced by labor in chunks, mostly due to great strife or revolutions (40 hour work week, child labor laws, etc.). Gains in efficiency experienced by capital are immediate and continuously accruing.

No, being beholden to a payments and banking system that can cancel you at any time, for any reason, with little hope for redress.

That you can't transfer large sum of money because money laundering rule. and you can't break it into smaller pieces either because that is called "structuring" and is a crime?

> you can't transfer large sum of money because money laundering rule

when did that happen? I can transfer as large an amount, provided i can prove the providence of said money wasn't from crime.

He is meaning “money” = cash, and in practice he is correct. The mere possession of large amount of cash is a crime - a crime charged against the money itself in a bizarre twist of the law to end run constitutional concerns - and one which year by year it is getting harder to fight.

In the pre-banking era you could get robbed, killed, taxed to death, made to fight in a war, etc. That's all still true, but now most of your resources (savings) will be stuck in a place where there is no need for violence to cause you to lose all of it.

The way we normally deal with all these problems as a society is to apply force (via police, courts) to ensure that everyone has certain 'rights'.

The problem today is that while you have certain rights, like fairly strong rights to private property interests in real estate and such, we generally have very weak private property rights in financial properties. This is a problem world-wide. We need to fix this in our countries.

They are proud people who want to defend their country.

> No piece of dirt or earth is worth dying for, ever.

If no one ever defends the dirt, the pieces of earth where you can enjoy a drink in peace and freedom will shrink over time as the aggressors will continue to gobble up land because of the lack of defending.

They keep moving forward, you keep moving back, until you have no where to retreat to.

Come back to this comment in a few years and think about whether something significant has changed for those people who did not sacrifice their lives for a meaningless battle.

People are more important than the state. If they are not ready to defend him, why should they be forced? You can offer money or other valuables in return, such as fame, a pension, or a position, but if a person doesn't want to, why should they do it?

> Come back to this comment in a few years and think about whether something significant has changed for those people who did not sacrifice their lives for a meaningless battle.

My family is from Eastern Europe: if people had not fought "meaningless" battles then the land would have been ruled by genocidal maniacs. As it stand my grandmother almost ended up in an oven.

My very existence is the result of the battles having meaning, that people fighting matters.

There is a significant difference.

Nothing will change for an ordinary person now.

At that time, the Genplan OST implied the almost complete extermination and the enslavement of a small number of the remaining people.

And also going back to the second part of the top commentary. At that time, people had a great motivation to defend their homeland and their loved ones. The survival of the country and the survival of the people in it were inextricably linked.

The current conflict has no such connection. The existence or cessation of the existence of the state is not related to the existence of people in it. Many of whom found life in a completely different country.

There were already volunteers, mercenaries, those who fell for a good salary. Why force those who actively avoid it?

Russia doesn't just "visit" your country. Lookup what Ruskiy Mir (Russian world) really means, basically your country gets subjugated by the Russians and I'm not talking about civilized or professional Russian forces - I'm talking about drunk and poor 20yo boys from a remote Russian villages that are now seeing the spoils of western civilization for the first time (do lookup what happened in Bucha, Kyiv suburbs in 2022 at the onset of invasion). Then of course the refusal of the Russians to recognize any other culture or language...the list goes on and on. So - yes, you could escape with a drink but then "If Not Me, Then Who"?

"I suggest bending over immediately and giving in to their demands" -- parent poster

and this isn't "just politicians", they're cleansing populations and relocating children

If politicians and the people who voted for them invade my country and hurt my family and friends I will be looking at revenge.

You have a very romantic view of Russian soldiers.

You have to be be proud, yes.

Interesting. So you put these into the project folder for Claude to follow?

I have zero issues with Claude improving from GitHub and feel like we should all help developers improve in using GenAI.

Eventually you can show Claude how you solve problems, and explain the thought process behind it. It can apply these learnings but it will encounter new challenges in doing so. It would be nice if Claude could instigate a conversation to go over the issues in depth. Now it wants quick confirmation to plough ahead.

Well I feel like this is because a better system would distill such learning into tokens not associated with a human language and that that could represent logic better than using English etc for it.

I don't have the GPUs or time to experiment though :(

Yes, but I would appreciate it if it uses English to explain its logic to me.

Source?

Most of the people I know doing local AI prefer SDXL to Flux. Lots of people are still using SDXL, even today.

Flux has largely been met with a collective yawn.

The only thing Flux had going for it was photorealism and prompt adherence. But the skin and jaws of the humans it generated looked weird, it was difficult to fine tune, and the licensing was weird. Furthermore, Flux never had good aesthetics. It always felt plain.

Nobody doing anime or cartoons used Flux. SDXL continues to shine here. People doing photoreal kept using Midjourney.

> it was difficult to fine tune

Yep. It's pretty difficult to fine tune, mostly because it's a distilled model. You can fine tune it a little bit, but it will quickly collapse and start producing garbage, even though fundamentally it should have been an easier architecture to fine-tune compared to SDXL (since it uses the much more modern flow matching paradigm).

I think that's probably the reason why we never really got any good anime Flux models (at least not as good as they were for SDXL). You just don't have enough leeway to be able to train the model for long enough to make the model great for a domain it's currently suboptimal for without completely collapsing it.

> It's pretty difficult to fine tune, mostly because it's a distilled model.

What about being distilled makes it harder to fine-tune?

AFAIK a big part of it is that they distilled the guidance into the model.

I'm going to simplify all of this a lot so please bear with me, but normally the equation to denoise an image would look something like this:

    pos = model(latent, positive_prompt_emb)
    neg = model(latent, negative_prompt_emb)
    next_latent = latent + dt * (neg + cfg_scale * (pos - neg))

So what this does - you trigger the model once with a negative prompt (which can be empty) to get the "starting point" for the prediction, and then you run the model again with a positive prompt to get the direction in which you want to go, and then you combine them.

So, for example, let's assume your positive prompt is "dog", and your negative prompt is empty. So triggering the model with your empty prompt with generate a "neutral" latent, and then you nudge it into the direction of your positive prompt, in the direction of a "dog". And you do this for 20 steps, and you get an image of a dog.

Now, for Flux the equation looks like this:

    next_latent = latent + dt * model(latent, positive_prompt_emb)

The guidance here was distilled into the model. It's cheaper to do inference with, but now we can't really train the model too much without destroying this embedded guidance (the model will just forget it and collapse).

There's also an issue of training dynamics. We don't know exactly how they trained their models, so it's impossible for us to jerry rig our training runs in a similar way. And if you don't match the original training dynamics when finetuning it also negatively affects the model.

So you might ask here - what if we just train the model for a really long time - will it be able to recover? And the answer is - yes, but at this point the most of the original model will essentially be overwritten. People actually did this for Flux Schnell, but you need way more resources to pull it off and the results can be disappointing: https://huggingface.co/lodestones/Chroma

Thanks for the extended reply, very illuminating. So the core issue is how they distilled it, ie that they "baked in the offset" so to speak.

I did try Chroma and I was quite disappointed, what I got out looked nowhere near as good as what was advertised. Now I have a better understanding why.

How much would it cost the community to pretrain something with a more modern architecture?

Assuming it was carefully done in stages (more compute) to make sure no mistakes are made?

I suppose we won't need to with the Chinese gifting so much open source recently?

> How much would it cost the community to pretrain something with a more modern architecture?

Quite a lot. Search for "Chroma" (which was a partial-ish retraining of Flux Schnell) or Pony (which was a partial-ish retraining of SDXL). You're probably looking at a cost of at least tens of thousands or even hundred of thousands of dollars. Even bigger SDXL community finetunes like bigASP cost thousands.

And it's not only the compute that's the issue. You also need a ton of data. You need a big dataset, with millions of images, and you need it cleaned, filtered, and labeled.

And of course you need someone who knows what they're doing. Training these state-of-art models takes quite a bit of skill, especially since a lot of it is pretty much a black art.

> Search for "Chroma" (which was a partial-ish retraining of Flux Schnell)

Chroma is not simply a "partial-ish" retraining of Schnell, its a retraining of Schnell after rearchitecting part of the model (replacing a 3.3B parameter portion of the model with a 250M parameter replacement with different architecture.)

> You're probably looking at a cost of at least tens of thousands or even hundred of thousands of dollars.

For reference here, Chroma involved 105,000 hours of H100 GPU time [0]. Doing a quick search, $2/hr seems to be about the low end of pricing for H100 time per hour, so hundreds of thousands seems right for that model, and still probably lower for a base model from scratch.

[0] https://www.reddit.com/r/StableDiffusion/comments/1mxwr4e/up...

Seedream is the only one that outputs 4k. Last time I checked that is..

MacBook Air then

Yeah, I think I'd be more successful with trying to convince them to move to a Mac.

Yes, this also seems bonkers to me.