> The ChatGPT maker is developing tools with the US Defense Department on open-source cybersecurity software, and has had initial talks with the US government about methods to assist with preventing veteran suicide, Anna Makanju, the company’s vice president of global affairs, said in an interview at Bloomberg House at the World Economic Forum in Davos on Tuesday.
> But OpenAI maintained a ban on using its tech to develop weapons, destroy property or harm people, Makanju said.
One step at a time! Next it'll be defensive technology only, then defensive weapons, then the best defence is a good offence. Before you know it, it'll be "wipe out the humans to achieve peace in our time."
In theory all our weapons are defensive. Hence the name "Defense Department". We will know to start worrying when GPTChat starts chatting about protecting its 'precious bodily fluids';)
One step at a time! Next it'll be defensive technology only, then defensive weapons, then the best defence is a good offence.
LLM chatbots pose no existential risk, because they are not autonomous. However, give a model an OODA loop and planning, and it could become an existential threat.
The problem with military applications, is that having an OODA loop which is potent enough to defeat a human being is basically required. Once multiple factions have such AIs, then those AIs will be in a mutual arms race and become ever more potent. Planning capabilities will also improve in such an arms race.
Before you know it, it'll be "wipe out the humans to achieve peace in our time."
If the AI arms race is very rapid and the improvement curve is steep enough, then humans might become as irrelevant as black powder arms on the modern battlefield. (And everywhere else)
Preposterous, sir! I have it on good authority from the marriage equality bros that slippery slopes are just fear-mongering. Why would schoolteachers all of a sudden be transing kids just because some tiny, tiny proportion of adults are getting visiting rights at hospitals? Moore's Law--also slippery slopism. As if computers will magically get faster next year just because they did the last n years!
Actually black Americans tend to vote strongly against child genital mutilation. If you had only listened to black voters (of California Prop 8 fame), the civil rights struggle of our time would almost certainly not be sterilizing people who because of their youth we do not allow to ride a bicycle without a helmet.
The article got updated since you copied this, it's now:
> The ChatGPT maker is developing tools with the US Defense Department on open-source cybersecurity software — collaborating with DARPA for its AI Cyber Challenge announced last year — and has had initial talks with the US government about methods to assist with preventing veteran suicide, Anna Makanju, the company’s vice president of global affairs, said in an interview at Bloomberg House at the World Economic Forum in Davos on Tuesday.
So they're working with the Defense Department because it's a DARPA competition that ultimately benefits the Department of Veteran Affairs.
Okay, but I'm not advocating that. My point is that they're being weasels. They're setting the stage so that they can work on things that anyone with a moderately flexible would view would qualify as "weapons", while hiding behind "it's not technically a gun or a missile".
I'm with you. I think weapons are important. I don't like double-speak, though.
Who has been unarmed exactly? In recent events I mean. Also, you are proving their point. The side who is killing the most civilians right now, in the past few months, is not at all considered to be evil by the USA.
The most obvious example would be Ukraine having given up their nuclear weapons following the fall of the USSR. It hasn't exactly worked out well for them.
Quite a number of studies suggest that chatbots are an effective tool for mental health support. Doesn’t need to be either or, but one could imagine scenarios where it may be more effective than a human mental health professional, e.g., 24/7 availability.
I do think there’s some nuance to that research though - just the act of typing out thoughts alone to a neutral party seems like it would be helpful, regardless of what’s said in response.
However, speaking personally - if I were suicidal, called an emergency line, and got connected to a computer instead of a person: that would feel like a brushoff and make things much, much worse.
My belief is that chatbots can be great for general mental health maintenance but are likely a massive liability for an acutely distressed population in the general case. I have zero faith in the US govt.’s implementation to respect that subtlety.
It's not going to stop there. They went full 180 degrees and now care about the money above all else. I don't trust them and hope more competition gets in the arena.
It's the first step on the path. Each incremental step is easy, and feels harmless. At first your drone only scouts. Then it gets used for actionable reconnaissance so a live feed is added. Then it's used for targeting so a laser designator is added. Then it's already there so it might as well launch the missiles.
Each step in isolation is logical and helpful and the end result makes a useful tool into an implement for killing people.
I've asked ChatGpt to do simple sysadmin stuff but half of the time it just made up new commands. I'm not sure how this will work with making systems more secure.
Great. The idea that we should take our lead in AI and ignore it's applicability for military applications is juvenile at best. I like living in the unipolar American world and I want it to stay that way.
I don't believe the person you replied to meant to convey that such a change will be positive or negative for him, but merely that the status quo will likely change in the coming decades
See the thing is, most places are in danger of being affected by various changes with regards to AI and other larger things. The game then becomes not "who's to be affected", but "who stands to lose more".
That's the conventional wisdom, but China is still dependent on the West today and it always has been, and assumes that the US and EU will stumble badly.
While understandable short-term - as someone from neither of those places: no, and bug off. But that's a short-term and emotional response. You want an arms race you'll get it. Eventually someone might press the red button indeed. Not too unlikely - and even then history will never learn the truth who really did it first (not that it matters).
As someone else pointed out above - this is going to be a surprising century for y'all promurica folks. Or for the the posterity, at least. Won't even be China to be blamed. Might not even be a state. Not seeing where this train is headed (and not just with the AI) - this is the biggest issue with the overall comprehension of the larger picture. But opening your eyes and learning to make sense of the world around isn't something you can learn by being lectured on HN, unfortunately.
It's a soothing feeling to have, though - this belief you're on the "good guys" team. Comfy. Also helps when you're arguing, doesn't it? OR when your passport allows you to travel to more places. Very nice indeed, but not a single a thought is usually given as to why, say, the latter, is the case. "Ignorance is our only ammunition".
DISCLAIMER: I DO NOT hate America. It's given this world a lot of good things and personally affected my life through art, books, movies, music, language and other things. I used to visit, but I think I could've never enjoyed living there. I used to think it's the American movies that perpetrate lies about how life really is, but then when you see it, you realize the lies is anywhere but in the movies (even the bad ones). Lies not by the people... Just lies. Fake. Vanity. Madness. And the only true thing I found in America was my one and only love. For which I'm forever grateful.
I'm guessing based on the OPs location, there is a very real possibility of being overrun by Russia if it weren't for America(or NATO). So this isn't some person shouting "America is #1" without having any idea of the outside world, it seems like a very pragmatic preference based on the alternative if the preference wasn't available.
My father's family was from a village which when invaded by Germans, seen half its population shot (probably worse than just shot, I imagine). Shall we go back and back and back in history, and when exactly do we stop so that the question of who's the good guy and who's the bad guy is finally forever resolved? Please tell us. And even if resolved, what then? What exactly is your proposition? Teach the bad guys your good ways? Good luck.
Let me guess, France? If Russians had come to "liberate" your village from the Germans, half of the population would have been deported to Siberia. You could argue that it's a better fate than getting shot, but nobody here is arguing for a nazi occupation either.
I don't think guessing things implied not to be mentioned leads to a meaningful conversation. There's a reason I never mentioned where I'm from or where my father's family was from - it's because it would create bias that right now you are trying to artificially induce by this guess. And no, they were very much shot and dropped dead... in the end.
May be. But words are words. We can't use deduction or metadata even to determine that. And who cares. It could've as well been written by a very real person.
That scenario is both prevented by and worsened by American primacy.
The previous - and very possibly future - President openly espoused Putin-friendly propaganda and threatened to leave NATO. European dependence on American power worked for many decades, but may leave them in a very tough situation now.
I am certainly no Trump apologist, but threatening to leave NATO was a classic "art of the deal" tactic to get some NATO countries to pay their fair/agreed upon share.
If Trump really is Putin-friendly, then Putin made a massive mistake by delaying his major military actions in Ukraine until Biden was in office.
I have no real proof of this but I suspect if Putin did delay his actions until Trump was out of office, it was because Trump was such a wild card that Putin might think that he could try to nuke the Kremlin if he didn't like what was going on in ukraine.
Putin delayed his invasion because COVID-19 was rampaging through the world at that time, and Russia had no access to an effective vaccine while being well...Russia, so it went about as well as you'd expect for their military readiness.
Disease killed more people on the frontlines in WW1 then weapons did, and it is the foolish military planner who doesn't try to avoid it (the Russian lines are currently dealing with an outbreak of Cholera in parts, which is another problem partly as a result of training quality - effective military's drill field hygiene practices because it's a direct contributor to combat effectiveness).
EDIT: i.e. when there's a novel respiratory virus about, nothing will destroy your military faster then massing for an invasion in one place.
At the point COVID was around and known, it would already be too late, because there would be less than a year left in Trump's first term. If I was Putin and I had a useful idiot in the White House, I would start planning that in 2016 before he was even inaugurated.
I'm not sure we are living in a unipolar world anymore. We are, at the very least, heading towards a multipolar world with the US, China, and Russia as the three great powers.
>We are, at the very least, heading towards a multipolar world with the US, China, and Russia as the three great powers.
US, EU, China, India.
Russia is a big country by size and resources, so it will be important, but it is a small (and getting smaller) country by population size and economy.
Their economy is actually expanding both in terms of GDP. And GDP PPP (now on par with Germany, which is in major industrial decline). It’s public information and pretty well covered by business and economic publications. Their biggest economic issue right now is an acute labor shortage.
I don't think anyone seriously considers Russia as an emerging superpower. There's a difference between being an international player and being a superpower.
Why not? Their economy is growing (despite being the most sanctioned nation on earth and cut off from SWIFT) and they are integrating with several other regional players. They are energy and food self-sufficient, have a relatively large and well educated population, launch payloads into space on the regular, export complex things like nuclear reactors. On the military side, they are one of the only that could sink an entire carrier battle group or an entire city in a flash.
Russia has a shrinking population. Their demographic distribution is among the worst. Their higher education system is a mess, and in most ways has actually declined since the collapse of the USSR.
On the military side, their ICBMs can still destroy cities. But their ability to take out a carrier strike force no longer exists at a credible level: some of the key links in the kill chain barely exist any more. They have no real expeditionary capability and can't project power much beyond their immediate borders.
Their population has mostly stabilized (still declining, but not at previous rates) and their demographics are better than many Western European countries and certainly better that the rest of Eastern Europe.
> their ability to take out a carrier strike force
A Project 22350 could absolutely disable a carrier strike group:
Their economy is not growing, they're just dragging people out of unemployment because they're in a war economy. That makes GDP numbers look good but in reality they're getting their citizens killed and using a bunch of production capacity to build vehicles and weapons that get destroyed a few months later, they're getting poorer.
Yes, their economy is growing and it's been widely reported. GDP PPP is now on par with Germany. And that's even after becoming the most sanctioned country on earth and cut off from SWIFT.
Regarding "unemployment" - they've got historically low unemployment and an acute labor shortage at the moment. Also very widely reported.
Yes, those are all results of war economies. It looks really good on paper but such generic numbers like GDP aren't that useful when comparing very different types of economies.
Russia has lost a significant portion of their export customers and is now losing a lot of productive capacity to keep the war going.
Their economy is not growing, that's just GDP being boosted on paper by a war economy, as usual.
Dragging a bunch of people out of unemployment to go die in trenches or to build IFV's that get blown up 2 weeks later makes your GDP larger but your citizens poorer.
Most of the things you mentioned are leftovers from the Soviet days, and are presently in decline.
> Their economy is growing
As the other commenters said, that's a war economy, plus some creative bookkeeping/regulation.
> and they are integrating with several other regional players.
The past few years have shown the CSTO¹, EEU², CIS³, etc. are a joke.
> They are energy and food self-sufficient
So are Canada, Brazil, Australia, Indonesia…
> relatively large and well educated population
Aging and dying. "Demographic crisis". Same for Japan, but at least Japan's doing better in most other metrics.
> launch payloads into space on the regular
"Regular", in the sense that the launch rate collapsed by over 80% in the late 1980s, and hasn't recovered⁴, even as the US and PRC have been exploding.
> export complex things like nuclear reactors.
So do Germany, France, Canada, the Netherlands— Most advanced economies, really.
> On the military side, they are one of the only that could sink an entire carrier battle group or an entire city in a flash.
Their flagship needs a tugboat following for when it bursts into flames⁵. They've built a grand total of 36 "5th-generation" combat aircraft, which are apparently less advanced⁶ than most countries' newer 4th-gen fighters⁷. They've even used up their stockpiles of tens of thousands of ancient tanks (at least the ones that hadn't already been rusted and looted irreparably) at a rate⁸ they'll never be able to replace.
Nuclear weapons aren't hard if you have a civilian nuclear energy sector. Rockets are old tech unless you're looking at non-chemical propulsion, reusable boosters, etc. Lots of countries could do those things within a couple years if needed, but we don't because they don't usually make enough sense unless you're already a superpower in other ways⁹.
Russia does a large number of flashy things, but it's for prestige, momentum, and autarky, and they can do less and less of it each year, rather than expressing any actual economic power.
They probably could have done quite well by focusing on tech/cultural, manufacturing, and resource exports. But wealthy nations won't buy from them now they've gone mask-off with the genocidal aggression, and their civil society is too corrupt to the core to fix itself.
The economic growth measurements are by Western institutions. Considering they're cut off from SWIFT and the most sanctioned and demonized nation on earth, they're actually doing pretty well (remember, they were supposed to completely collapse like a house of cards).
> CSTO¹, EEU², CIS³, etc. are a joke
BRICs combined GDP is larger than the G7, and growing. Sans Russia and China, that's where the future population growth (and economic opportunities) are.
> Canada, Brazil, Australia, Indonesia
Brazil and Indonesia are absolutely up and coming major powers.
> Aging and dying. "Demographic crisis".
While still declining, their demographic situation has largely stabilized. It's better than the rest of Eastern Europe and even better than many Western European countries.
> "Regular", in the sense that the launch rate collapsed by over 80%
In the 1980s it was the USSR. We're talking about just Russia, a much smaller population and economy. And they launched at least 20 large payloads just last year, including a number of military sats.
> Germany, France, Canada, the Netherlands
Yes, you're making my point.
> Their flagship needs a tugboat
An old aircraft carrier is not their "flagship"; NATO ships also age out. Look into Project 22350 and Yasen class subs. Both carry Zircons and could disable a CBG.
> They've built a grand total of 36 "5th-generation" combat aircraft
If you're talking about the Su-57, it only went into serial production this year (2024) and their prototypes already have more air-to-air kills than the F-22 (via the R-37 which out ranges anything in NATO inventory).
Remove Russia from that list…the Ukraine war has shown that the Russian military are chest-beating clowns tainted by massive corruption and incompetence.
I kinda suspect China is the same, but we’re yet to see, and hopefully never see any invasion of Taiwan..
This is what it is to be in a multipolar world though.
I agree with your implication. The military forces of Russia are not paper tigers. Don't believe the hype. But at the same time, you don't want to fight proxy wars? Ok. Great. Then don't screw with other nations. If they hadn't messed with Ukraine, they wouldn't be fighting a proxy war.
The system works, again, because each of the major powers are comfortably confident in their ability to destroy the other 3. There is therefore, nothing to stop us from fighting a proxy war with Russia, because Russia will never use hard power to do anything about it. Unless Russia is suicidal, and they aren't, they will not escalate to a direct conflict with us.
No one is suicidal, but escalation don't always work according to logic. It's people making decisions, and people are emotional. Another thing to consider is, if you're the one up top, perhaps pressing the red button and living in a nice warm bunker with staff serving you is much better than being ousted, voted out or accused of corruption and facing the probability of going to jail (depending on the country). So I wouldn't be so sure we're immune to the global war just yet.
> The military forces of Russia are not paper tigers.
If there's anything that their gloriously failed sneak attack on Ukraine has shown us, it's that yes, yes they are. Imagine if the most advanced weaponry they were facing wasn't small amounts of slow-coming ancient hand-me-downs employed by people who have barely even trained to use them . . . .
Ukraine would of course be toast without arms and intelligence from the West, but neither have any interest in starting or perpetuating the war. Russia's folly is convenient for the West though.
>neither have any interest in starting or perpetuating the war
I don't think this is strictly true. Cutting Russia of from unimpeded Black Sea access and draining their resources in a (for America, cheap) proxy war are both in Western interests.
While any war, as Hemingway wrote "no matter how necessary or justified - is still a crime", without US, even the events in 2008 and 2013 would not take place simply because they would not had been necessary due to Ukraine staying in the Russia's sphere of influence. This is an unwinnable war for Ukraine. It is bad for Russia and certainly puts a dark spot on its history, but if there is at least one large and powerful country without dark spots in its history, please tell me which one it is. Human history is catalogue of crime.
USA is a large closed economy. China is a large open economy. I’m simplifying, but the bottom line is China definitely doesn’t want a Cold War scenario.
I think so too, but we should also understand that authoritarians are not rational.
Before February 2022, I would have bet a big sum on Putin never authorizing a full-scale Ukraine invasion. There was just too much to lose and too little to gain theoretically…yet he did it anyways.
Putin was lied to about the capabilities of his forces. Now we’re seeing the same thing with Xi, except heads are rolling _before_ the invasion gets underway.
No, they set out to take the Kiev in 3 days. They never got close. It was a spectacular failure based on the power imbalance between the two countries.
But I guess you are right on one point, Russia knew how weak their armed forces are and doubted themselves so heavily they never said it out loud. It was just the rest of the world that finally saw what a paper tiger Putin's forces are. This really doesn't help your argument that they are in the running to be a world power one day.
"The convoy stalled 8 days into the war, at approximately 30 kilometers (19 mi) from the center of the city of Kyiv; as of 7 March 2022, according to U.S. defense officials, the column had not moved at all for a few days."
Isn't it interesting how those "stalled convoy" dates line up with these "peace" negotiations (and that Ukraine didn't burn them Road of Death style when they very easily could have)?:
Well, it is limited in scope and they literally stated their territorial objectives when they set out. Full scale is mobilizing your entire economy towards the war effort and calling up millions. And it’s taking out the opposing sides military and civilian leadership and all of their critical infrastructure. That’s full scale. I really hope we don’t see full scale.
I believe they "started" with claiming that statements that the they were about to invade Ukraine were absurd lies, followed shortly later with claims they would control the entire country in three days along with some maps that included Moldova, followed shortly thereafter with lots of nuclear saber rattling when things started going south.
There have also been dozens of assassination attempts against Ukraine's civilian and military leadership, and against Russia's as well, some of which have been successful. The fact that most attempts don't work out doesn't mean it's not happening.
We really aren't, it's just very hard to convey the scope of US military supremacy. But a short version would be: the US is so militarily supreme that a huge chunk of it's populace regards it's failures in occupation and nation-building (resulting in democratic, western-values states) as military failures.
This is both a problem (i.e. the reason that doesn't work is because it's pretty explicitly not a military problem) and kind of amazing.
The US military just spent 20 years in Afghanistan, a landlocked country on the other side of the world with little economic value, and only left because our politicians chose to leave. We could have afforded another 20 years easily with hardly a blemish on our economy or military readiness for other conflicts. Finally accepting the fact that Afghanistan had very little strategic value to the US was hardly a military defeat.
No that is absolutely not the definition. You win a war when you meet your political goals. The military didn’t lose, the grand strategy changed.
This isn’t sophistry, war is just extension of politics by other means. The vast majority of unfixable problems the US faced in Afghanistan were political, you can’t kill your way to a functional multiparty democracy.
No. I’m saying the military didn’t lose the war, American and Afghani politicians did. And not because we decided to leave, but because there was no strong strategic reason to be there after the primary casus belli was accomplished: the elimination of Al-Qaeda from the region.
Unless you are looking narrowly at nuke counts, Russia is not a great power. It's economy is propped up by oil and gas, and that's in question going forward, it is inefficient and outside of primary products it doesn't produce much of anything anyone wants. Its military has been exposed as a paper tiger and even the nukes can't be guaranteed to work these days.
China has the economic and manufacturing clout and is building up its military, but outside of the 'Global South' and various needy countries it has little soft power.
The US has stumbled somewhat mostly due to the overreaction to 9/11 and Bush's disastrous presidency, followed by the overcorrection of the Obama and Biden presidencies. But it still has the biggest and most powerful military, and NATO has been shocked out of its long complacency.
What the US has going for it is the possibility of renewal and reinvention, that is its strength and its enduring power. The US will keep on innovating, hopefully also in the political realm, and will retain global leadership. That's assuming that the Trump led Republican nihilist suicide cult doesn't destroy it first.
The Russian economy has been diversifying away from oil and gas for a decade and is close to the US in terms of percentage of the economy. They produce about as much steel as the US with less than half the population, a wide variety of other vital raw inputs. They’re a massive food producer and also export complex things like nuclear reactors, rockets, fighter jets.
Russia can punch above its weight because it has more people willing to die for the country in physical battle. Might makes right in realpolitik, especially in a multipolar world.
But you're right. We definitely don't live in a unipolar world. But that's a good thing. More people around the globe enjoy the benefits of peace when each of the major powers are all comfortably confident in their ability to annihilate the other 3.
There used to be coups all the time in Latin America. Not so much anymore. All of Asia is rising and enjoying the benefits of peace. And don't even get me started on how much better off Africans are with the ascendancy of China than they ever were under the US and Russia.
The world 70 years ago was a bleak place for the global south. War, famine and disease ruled the day. Today, in any dimension of human development along which a person would care to take a metric, most of the global south are far better off.
The EU is vastly underrated, sadly by the EU itself. If it were a federation, it would be the largest economy and would wield substantial economic and soft power, and potentially military power too.
A strong US and EU in alliance would be dominant combination.
If the EU got their act together, and were able to wrest Africa from the US and China, they would be the dominant power on the globe. I don't even think they would need the US.
If the US doesn't do it then some other superpower surely will. I guess it doesn't hurt to explore what AI is capable of in a military context, so you know what you're up against.
Altman was of two minds about handing OpenAI products to Lynch and Carter.
“I unabashedly love this country, which is the greatest country in the world,”
he said. At Stanford, he worked on a darpa project involving drone helicopters.
“But some things we will never do with the Department of Defense.” He added,
“A friend of mine says, ‘The thing that saves us from the Department of Defense
is that, though they have a ton of money, they’re not very competent.’ But I feel
conflicted, because they have the world’s best cyber command.” Altman, by instinct
a cleaner-up of messes, wanted to help strengthen our military—and then to defend
the world from its newfound strength.
Why this is surprising? I mean wouldn’t any military in the world want to benefit from new technologies, especially if it was developed in its own country?
It's "surprising" because it was an explicit part of OpenAI's charter to not allow their tech to be used by the military, but to anyone with half a brain I don't think it is ACTUALLY a surprise.
To have their cake and eat it too: Many top minds have this pesky not wanting to be complicit in killing people thing... so say that you super promise not to work with the military can sometimes trick those people.
Hitler wasn't on track to 'get the bomb' before the end of the war.
So it wasn't an America or Germany situation, in fact nobody on the planet was close to 'getting the bomb' in 1942. Though not for lack of trying. The Manhattan project was a stupendous effort compared to anything else done to move science forward by brute force up to that that. And short of the space race nothing like it has been seen since until the current effort to bring AI into the world and in some ways it still pales in comparison.
How long until enemy soldiers wear camouflage? How long until decoy targets are set up? How long until the enemy tries to blend in with the civilian population? How long until they start wearing body armor? How long until they start looking at EXIF data on Facebook pictures and mortaring those GPS coordinates?
Yes, war is a game of cat and mouse. Notice how no exploit or countermeasure is definitive.
why do we think these things will be programmed to accept external prompts anyways? seems like a really bad way to implement this thing. like developing a death star which is nearly invincible all except for this one little bitty flaw
Seems like a change in their previous stance, defense-tech and working with the gov is "in" now. I would be so curious to know if this was an Altman initiative that would have been shot down by the previous board.
Although I am still in awe of OpenAI's achievements and greatly respect many people there, they claim the record for the company that has most rapidly fallen from my own personal grace.
When they were founded, I was hopeful that the capped profit structure could be a way of preventing enshittification, where the winnings of these mega successful companies are more equally distributed between shareholders, employees, and the public.
Instead, they seem to have basically betrayed all of those founding principles. Now they're just another tech company.
nah, I meant more in general. Any of us running uBO or other things in our browser to block things are basically futilely resisting. there's plenty of ways to fuck with the system that doesn't
> But OpenAI maintained a ban on using its tech to develop weapons, destroy property or harm people, Makanju said.
At face value, seems like a good thing.