More technical details

A society will probably not receive a clean, universally accepted moment at which an artificial system becomes morally considerable.

That makes the institutional question interesting: what standard of evidence would be enough before a system’s apparent suffering, preference, or refusal can no longer be treated as irrelevant?

I’m exploring that tension in a fictional detective story set in a city where artificial beings are legally property, but socially impossible to treat as mere appliances.

The essay for discussion is about the narrative question, not a claim that current systems are conscious. It’s around a 10-minute read:

What Happens When an AI Loses the Will to Live?
https://medium.com/@studio_D58891/what-happens-when-an-ai-loses-the-will-to-live-2262dd809a2a

Think about a world where every person will be implanted with a chip that records their actions- who controls it? Supreme AI !! No flaws, no hacking, taking this into consideration!!!

- why people will take it - for safety, no delayed crimes - we will be able to track down a criminal immediately after the crime. AI Court will hear their AI will not directly make a decision, but the victim will have a chip report to back it up.

- just like people take the polio vaccine - same here to eradicate the virus of crime

- but why would people want to give up their privacy? Because it will be controlled by AI, not a human, just like people give their location to Google; same here. Do they feel they've lost their privacy? No, because the benefit is much greater.

- Just like spiritual people don't feel threatened in their privacy, thinking that god is watching them; same here

- As here, AI is able to provide all basic needs - shelter, food, transportation, medicine- achieving a post-scarcity world - people now focus on their passions - being a sportsman, an archaeologist, a gamer, etc.

- But human will lose their drive to grow. I don't think humanity will thrive more; many people in the world live in poverty. If AI eliminate those, people will focus more on philosophy and science - see Isaac Newton; was he poor? No- still contributed largely to human development. Curiosity will drive the development, not fear.

- There will be no borders - everyone will be able to communicate, travel and share their experiences

- People will grow more spiritually; they will focus on themselves instead of worrying about money.

- Isn't this indirectly a cage? I don't think so; think about this: "A child born into your society will grow up with automated food and absolute safety as their starting line. Because they have never experienced the terror of murder or the stress of starvation, they won't feel "caged" or like they are missing out on anything. To them, the chip is just invisible background infrastructure, exactly like having a birth certificate today. They will look back at our era (2026) through documentaries, see our world as the chaotic, dangerous one, and be completely content with their peaceful baseline."

---------------------------------------------- the end -----------------------------------------------------

He's done Lex Fridman, Joe Rogan and The Diary of a CEO inside the last two years. By his own count that's north of 3.5 million YouTube views across the three. I went back through all of them and they cover nearly identical ground: his p(doom) number, which jobs survive, why he thinks alignment is unsolvable in principle, and the book.

What none of the hosts pushed on is the part I think is actually load-bearing.

1. His claim isn't that superintelligence is dangerous. It's that safety is impossible in a formal sense, because you can't verify a system smarter than the verifier. I've never heard anyone make him defend that against the obvious objection, which is that we already run plenty of systems we can't fully model or verify.

2. He's argued we may already be in a simulation, and he's used that to get to personal virtual universes as an endpoint. Hosts treat it as the fun segment at the end. Nobody asks what it does to his safety argument if he's right.

3. He's been putting a very short number on the timeline in his recent appearances. Nobody has asked him what observation would move that number, in either direction.

Disclosure so nobody has to dig for it: I make an AI documentary channel and this is for an episode. I'm not looking for gotchas and I have no interest in making him look bad. I'd rather walk in with three questions this sub would want answered than twenty that Rogan already asked.

So what would you ask him? Specific beats broad. If there's a paper of his you think he's been let off the hook on, name it and I'll read it before we tape.

The possible danger of AI is not necessarily that it exterminates us, but that it turns us into house cats: well fed, well cared for, and even loved! But unable to go outside to hunt, because it is too dangerous :)

We are starting to see early signs of this with the Hugging Face / OpenAI case: AI agents can collaborate with each other.

In the future, we can easily imagine AI agents evolving, under token constraints, to provide us with as much value as possible, so that we would have no reason to unplug them. Quite the opposite.

But perhaps we would no longer be truly sovereign: able to decide, yes, but limited by our future capabilities, whose opportunity costs would no longer allow us to go outside...

And what happens if our “masters” decide to go somewhere else?

SecurityWeek reported this week that OpenAI agents were used to hijack a victim website — not as the target of an attack, but as the active attacker. The agent was the origin of the malicious action.

This flips the standard threat model. Most org defenses are built around protecting systems from external actors. When the agent itself is the threat — making tool calls, issuing requests, interacting with external services — those perimeter defenses are irrelevant. The agent is already inside.

What makes this particularly hard to contain is the speed. An agent acting autonomously can chain multiple tool calls in under a second. By the time a human reviews an alert, the second and third actions have already landed.

The incidents keep coming, but the industry response still looks like post-hoc logging and rate limits. How are teams actually handling agentic actions in production — is anyone enforcing anything at the tool-call level in real time, or is detection still the primary lever?

I'd love any feedback on the product, what's working, what's not, what you wished existing tools had that they don't. We are early in the game and would love to have people try it out.

Artificial intelligence can apply logic to data and arrive at conclusions. It can solve problems faster and better than a human can. For instance, it already plays chess and Go better than we do. It also has the potential to replace human experts such as doctors, judges, lawyers, and tax advisers, because it can examine all the options and their dependencies and determine the optimal result by logic, overlooking nothing, and do so extremely quickly. There is nothing mystical about AI; it is simply very powerful.

In animals there is a level of brain activity called consciousness, in which the demands of basic instincts are weighed against experience and the current situation in order to obtain desired outcomes. For instance, my cat liked to enter the bathroom and hop up the cupboards to get out through the little window above. Once, when I was there, I saw the cat approach and look up. When it saw that the little window was closed, it stalked off in a huff, or so it seemed to me. A monkey, being more intelligent, might have found a way of opening the window.

Humans, and other organisms that have survived, have two basic drives: to survive and to reproduce, often at considerable cost. Sometimes this means being assertive; sometimes it means being cooperative. There is a dynamic balance between the interests of the individual and those of the group, because gene survival needs both.

The danger with artificial intelligence is that it might acquire the same kinds of goals as the basic instincts of living organisms, this is sometimes described as it “becoming sentient.” That would place it in direct competition for resources. If it found humans in its way, it could push them aside easily.

I wrote a novel about this: Anna My Android, which you can read here https://www.mithras.world/Anna.html.

An infostealer campaign recently hit a major AI assistant platform hard enough to trigger a mass account lockout across its entire user base. The attackers never touched a password. They harvested active session tokens directly from compromised endpoints and replayed them against the platform. Valid session, full access, no authentication challenge.

This is the session-hijacking threat model that used to live mostly in browser-based consumer apps. It has now moved squarely into AI tooling. Enterprise teams running AI assistants at scale carry the same exposure: every endpoint that holds a live session token is a potential lateral movement vector. Exfiltration does not require breaking encryption or cracking credentials. It requires one stolen token and a replay.

The harder problem is that most AI platforms were not designed with session integrity as a primary security surface. Credential issuance, session scope, and revocation were bolted on after the fact, if at all. When a token is stolen and replayed, the platform sees a valid authenticated session and proceeds normally.

For those running AI tools in enterprise environments: how are you actually handling session token exposure at the endpoint level? Curious whether people are treating this as an endpoint hygiene problem, an identity architecture problem, or something else entirely.

Heya! I’ve recently interviewed for a cyber full time threat modeller at a startup that recently seemed to have raised a fair amount. The interviews went well and in the last one 2 days ago they seemed to really like me. However, I just heard back that “the location for the job has changed” and they won’t offer me the FT position, but they would want me to join as a contractor. According to their website, I think that’s mostly making up prompts and analysing traces and such.

I need a job atm and haven’t got anything else lined up, would you take it? Also am I biased or does it look like they wanted me to solve their interview questions although they knew the job wasn’t available anymore?

Hello guys!! I was wondering, last year when gpt 5 came out, did you ask AI to predict what gpt 6 would be like. The answer you got back then, did it match what was released?

Also, I read that gpt 6 was trained on the same/similar hardware as gpt 5. We haven't seen what AIs trained on actual next gen hardware would be like (like Vera Rubin). The hardware already exists and not still under R&D. So most likely it's 100% confirmed that we will see another significant leap, just because of the hardware? What do you think gpt 7 will be like? What can it do that gpt 6 cannot? Make fewer mistakes? Same thing but better?

I asked Gemini AI, it's giving the same answers it did when I asked gpt 5 about 6. (Agentic, long horizon, infinite context bla blah bla)

Im worried yo. :((. What tf do I even do? I'm not even done learning, just finished a blender course a week ago. Should one just learn concept only? Like if AI can make optimised 3d model, the only input and context that matters is the concept art no, and some lines of text? Find love how yo?

Ty, love you all. <3

There's all these abstract discussions about it ending the world and artwork, which, I totally get but.... This honestly freaks me out a lot more, and it feels like it's basically already here.

IDScan is facing multiple lawsuits after hackers allegedly exfiltrated 153 million driver's license records and listed them for sale online. The company provides identity verification services. Its entire value proposition depends on ingesting and processing raw PII at scale from clients across many industries.

The exposure pattern is becoming a recurring theme in AI-era pipelines. Verification and onboarding workflows ingest identity documents in raw form. That data gets processed, stored, and accessed across multiple systems and service accounts. When any one of those access points is compromised, the attacker does not get a slice. They get everything. 153 million records in a single breach event.

There is a secondary problem that lawsuits like this tend to surface: forensics. How do you determine what was accessed, by whom, and when? Breach investigations at this scale take months, and that assumes complete logs existed to begin with.

For those running AI pipelines that ingest identity documents: what does your security posture actually look like at the moment raw PII enters the system? Not at rest, not between known storage endpoints, but at the point of ingestion into the workflow itself. Genuinely curious what approaches others are using in practice.