r/singularity • u/BrennusSokol AI please take my job • 21h ago
AI The president of ARC Prize is actively soliciting ideas for games that Astra hasn't been able to play
•
u/TrainquilOasis1423 21h ago
A non blocking goalpost I have for AGI is it can beat every videogame. It's not the most important thing, but anything smart enough to solve real world problems like a human should be smart enough to figure out most or all videogames. Doesn't need to be the best player or anything just beat the campaign at least.
•
u/Quarksperre 18h ago
Especially new games. The issue with older games is that it has for every popular games been trained on millions of guides and whatever. A normal gamer can figure out a normal new game pretty fast without a guide.
And the second thing is exploitability. Neural nets in general, not only LLMs, are pretty exploitable
•
u/darkestvice 20h ago
Yes and no. I think you're conflating AGI with raw speed and awareness. There are many video games out there that require crazy high reaction times. There are also plenty of games that no one can win at without significant muscle memory from sheer repetition. Hell, there are plenty of games that most humans will never be able to beat. And while I don't doubt that AGI can get there eventually, I think it's too high a bar to expect from something we'd attach the AGI label to.
On the other hand, all turn based games for sure should be winnable by AGI. Because that indicates awareness, adaptability, and the ability to plan for the future, all of which would be hallmarks of AGI.
•
u/Quarksperre 4h ago
Would be interesting to test it on a full campaign of EU4 or even better EU5.
Its pausable and relatively slow but has a high complexity overall.
•
u/Perfect-Campaign9551 11h ago
It would have to have a beast of image recognition. Don't see that happening where it could run that fast
•
u/PsychoticDreemurr 21h ago
Osu! Is most definitely "the" game for real time/latency. Geometry dash if you want to add a more visual element
•
u/Effective-Painter815 21h ago
Osu! is famous for being where NeuroSama, the AI powered Vtuber originally started from. That AI still can absolutely demolish in that game.
•
u/ErmingSoHard ▪️Agi never with LLMs... Or? 20h ago
She was a specialized model though
•
u/Effective-Painter815 20h ago
Vedal (Creator) put her through it again recently, actually the twins which was the telling part. Neuro can still talk and do Osu! with a high level of skill... okay, so a Local LLM instructing the smaller Osu! model right?
But then Evil had a go, messed it up, wasn't as capable and threw a temper tantrum because she wasn't able to perform as she knew she (Neuro) should have been able to. So it looks like Evil has received training that has degraded her Osu! abilities. Implying that it's not a separate sub-model but an learned skill in the main models.
[For those that don't know Evil is a modified clone / duplicate of Neuro but with different personality prompts and then later different training / code changes as time went on.]
•
u/Antiprimary ▪️ 12h ago
Ok then you could have a new AI that uses the specialized model for OSU then astra for everything else and package it as one model. We need to find games that no model has yet been able to handle.
•
•
u/ThisGonBHard AI better than humans? Probably 2027| AGI/ASI? Not soon 18h ago
I dont see how it is a good at all. A special, small latency focused model would 100% clear the game. You can probably do it without AI too.
This is just a function of how fast your hardware is.
•
u/PsychoticDreemurr 18h ago
A TAS can complete those games, but a TAS can't solve equations.
A calculator can solve math problems much better then LLMs, yet we're interested in LLMs being able to solve them.
This isn't just testing the capable of being able to play a game. It's about being able to get such high scores on intelligence based exams while also being able to be extremely quick. And consistently so.
The image specified real time/latency, and I pointed out the best games for this task. Other games like fps and strategy games are also great tests, but they aren't the pinnacle of latency testing.
•
u/ThisGonBHard AI better than humans? Probably 2027| AGI/ASI? Not soon 18h ago
GPT 6 could write the TAS for that.
My point is that there is nothing in OSU besides latency. It is a deterministic game, and when you can think at faster than a frame, a solved game.
•
•
u/Hot_Glass_6301 20h ago
Geometry dash is almost entirely deterministic (ik some levels add random elements) and I'm sure it could already be beat using 2016 RL tech, possibly much weaker than AlphaZero. It's only hard for humans because it requires extreme timing precision but that's not a factor for computers. I mean sure it could be a benchmark for real-time zero shot discovery if you play under a contrived ruleset, but otherwise the model just has to spit out the series of keypresses
•
u/PsychoticDreemurr 20h ago
Geometry dash levels are as deterministic as every test based benchmark is, so I don't quite see an issue.
Also the thing that makes it impressive is if an AI is able to do it while maintaining its current intelligence. Of course a TAS could accomplish this, but no TAS could also do what LLMs are currently capable of.
This is a test of speed, as well as sight. Not skill.
•
u/Efficient_Loss_9928 14h ago
sounds like it just needs to go through the game to fail once, record the locations of each keystroke, and write a python script to play it?
there has to be some kind of randomness to it, if it is just a static level, AI will always beat it.
•
u/PsychoticDreemurr 14h ago
Technically it can also write a TAS for any game.
Or a program that automatically clicks the buttons as needed.
It's just a test of latency and image processing
•
u/Efficient_Loss_9928 14h ago
You cannot with starcraft with a real human opponent. Because the next round would be different.
•
u/PsychoticDreemurr 14h ago
That wouldn't provide a fair testing environment as each player will play differently, therefore making the test easier or harder arbitrarily. Even if it's the same person they'll have completely different skill levels across days, moods, times, etc.
•
u/Efficient_Loss_9928 12h ago
You can have a statstically confident score by running thousands of rounds with players at different levels.
•
u/PsychoticDreemurr 12h ago
How do you propose handling the logistics? You need to get that many people each time you want to perform a test, take into account those who drop out, deal with anyone that obviously didn't play properly.
You can't just run it on a games public matchmaking either, good luck trying to get permission from a publisher for that.
•
u/Efficient_Loss_9928 12h ago
not my problem, Greg need to figure this out.
•
u/PsychoticDreemurr 12h ago
You said my example wasn't great and the reasoning you provided is impossible under realistic conditions. You can't just say "not my problem" without acknowledging the fact that your reasoning is invalid.
•
u/Efficient_Loss_9928 11h ago
Yes, I said your example isn't great and provided an alternative. Not saying my alternative is practical. Just my reasoning why theoretically it is better and can theoretically be achieved.
I still think your example isn't great because they can achieve perfect score easily.
→ More replies (0)•
u/RobbinDeBank 20h ago
The visual capabilities of current models fall quite far behind the text capabilities, even after all the recent advancements. Their text abilities are so good that they can use coding and tool use to alleviate their vision problems. That means latency sensitive visual reasoning should be what challenges them greatly (self-driving cars are actually very good at this, although they are a much more narrow system).
•
u/The_OblivionDawn 21h ago
It's not real-time, but I wonder how Astra would handle a game like Stellaris
•
u/NoCard1571 21h ago
Probably very well. The fact it can beat 8 antes in Balatro is a lot more significant than I think a lot of people realize. I imagine any game that requires strategy is right up its alley.
•
•
u/sadphilosophylover 20h ago
eh imo balatro is quite well formed one to do
•
u/NoCard1571 20h ago
Maybe, but that doesn't change the fact that it's an unprecedented achievement for a generalized AI. Especially when you consider that probably 95% of humans couldn't do it without a lot of practice ;)
→ More replies (1)•
u/beekersavant 20h ago
I think Hades would be interesting but then the machine would just reaction time the game to oblivion. It has to be a game in which perfect reactions don’t matter. Aimbots are a real thing and it gets to aimbot. So shooters and Skill shot games are out.
•
u/Poupulino 20h ago
Make it play any Bethesda game in its release day status. If it can play around the endless bugs I'd be extremely impressed.
•
u/LocoMod 21h ago
Satisfactory
•
u/kikal27 20h ago
That's the answer. Not because is a "farm and plan" game, but because you as a player need 3d coordination, manage resources, be efficient and be able to be truly creative solving the next problem. I wonder how much time will pass until an AI could truly master not a plan but an effective-decision making on the fly.
•
u/Glittering-Toe-1622 8h ago
Those games have a lot of math in them and AIs are good in math.
Places where they struggle are abstract topics. Not sure what a good example is, maybe Elden Ring. The lack of direction and those abstract quests is where it should struggle.
Saying this as someone using AI in software development.
•
u/Concurrency_Bugs 21h ago
Here's a thought: Since many models use a sort of swarm of agents, what if they made a Battlefield benchmark. One "commander agent" who oversees the battle and issues commands to agent squads. Pit frontier model swarms against each other.
•
u/terrarialord201 20h ago
There IS (or at least there was) a commander mode in BF4, which you could play on your iPad. It let you issue tactical orders and call in cruise missiles on spots. Something like that could definitely be replicated, and it would be hella interesting to see
•
u/jakobjaderbo 18h ago
Wouldn't surprise me if some guys in the military are actively working on something similar.
•
u/Commercial_Monk_9736 21h ago
It more or less can play Civ 6, I had it play through 30 turns. It's just expensive to keep running. Doesn't mean it's good enough to beat a human but it can play the game.
•
u/ShelZuuz 20h ago
Ooo... How do you do that? I would love to play a modern AI instead of their idea of difficulty just being "Give the AI more resources".
•
•
u/Swimming_Gain_4989 21h ago
League of Legends and Dota are my "this is actually AGI" litmus. You need reactions, planning, strategy, collaboration, and both games have a rediculous amount of depth. Additionally the meta changes every 2 weeks so it's difficult to pretrain.
•
u/Leafsnail 20h ago
I mean they did beat pros with OpenAI Five in 2018. Wouldn't be that hard to pretrain given it's something they already solved
•
u/Swimming_Gain_4989 20h ago
Five was cool but not even remotely the same as a general agent playing Dota competently. It was a specialized model that could only play Dota, it played via direct API access, faster reactions, more precise clicks, perfect synchronization with it's teammates and a ton of items / heros were disabled.
If you could drop GPT-8 or whatever into ranked queue and have it reach Divine / Diamond I'd consider that AGI.
•
u/nick4fake 19h ago
How general human with no experience is playing competently?
•
u/Working_Sundae 18h ago
It's a fundamental AI problem, the one example that's often discussed, some can take driving lessons for a week or two and start driving like any other person would yet there have been billions poured into autonomous driving for the past 15 years with all the suits like radar, lidar, multiple cameras and technologies like sensor fusion and AI deep learning, and yet we still haven't achieved L5
And atleast in robotics they have started to target rapid learning with few examples, ofcourse it will extend to others as well
•
u/trifkograbez 20h ago
Extremely restrained the pros to the point it was moot tho. An actual game were the conditions are different is a totally different beast.
•
u/Glittering-Toe-1622 8h ago
Open AI destroyed almost ever top player 1 on 1 mid. Not sure if there was a 5v5 match but it was scary good back in 2018
•
•
u/staplesuponstaples 20h ago
The meta slightly shifts every 2 weeks. A 3-year-old benchmark for a perfect player would still incinerate any living human today since it would have perfect mechanics and understand what the game was generally about. Slight changes in gold and strength and champs would lead to some miscalculations and errors but as it stands, humans play with far more error.
•
u/Swimming_Gain_4989 20h ago
Should've said the meta CAN shift every 2 weeks, you're right it's not common for it to swing that much patch to patch.
Completely disagree on the 3 year old 'perfect player' beating any living human today though; matchup knowledge is too important. For example, Nasus is so broken right now that he has a 50+% winrate and 30% banrate in CHALLENGER wheras 3 years ago he was completely unplayable in high elo.
All that said I would factor in human level reactions and mouse accuracy as part of the benchmark.
•
u/MINECRAFT_BIOLOGIST 19h ago
There are two sides to this argument, and it basically boils down to whether you think a living peak human can beat a "perfect player" in a situation where the "perfect player" miscalculates the new strength breakpoints (e.g. which lvl 6 all-ins Nasus wins now vs 3 years ago) and the human player attempts to snowball that advantage.
I think this is too vague to answer, especially if we're defining "perfect player" as something like OpenAI Five or something like a theoretical AGI, lol.
•
u/Swimming_Gain_4989 19h ago
It's for sure too vague, only thought about it seriously because Nasus just got turbobuffed and he is one of those champs who has 0 counterplay if you make the wrong decisions early and give him a kill + plates.
•
u/MINECRAFT_BIOLOGIST 19h ago
Yeah that's true, it was a pretty crazy choice, speaking as someone who usually looks at the balance team quite favorably.
•
•
u/Shaod 20h ago
They solved that almost a decade ago.
•
u/trifkograbez 20h ago
That was with API access not like current astra uses the actual software with computer vision.
•
u/_hisoka_freecs_ 21h ago
Baba is you
•
u/terrarialord201 20h ago
Isn't Baba just visual programming? Ai could do that pretty well.
•
u/BrennusSokol AI please take my job 20h ago
Some of the puzzles it includes have a lot of subtlety that need abstract reasoning/abstract language
•
u/razorbeamz 9h ago
Also there are thousands of hours of people on YouTube solving the puzzles and tutorials online so it already knows the answers.
•
u/PayPalModApk 21h ago
Minecraft Hardcore is obvious answer. Requires real time latency to fight mobs and ender dragon, as well as strategizing and planning since every world is random, and lots of preparation. Plus if it dies it has to start over so it can't just brute force it.
•
u/Swimming_Gain_4989 21h ago
Meh, there aren't adversarial agents or agents you have to collaborate with. Speedrunning that game is pretty flowcharty.
•
u/staplesuponstaples 20h ago
The randomness is less an issue since you can have as many agents as you want. Just have 100 instances going at once and you'll sample size away the randomness.
•
u/NoCard1571 21h ago
Yea I think this would be a good one. Especially if you give it a list of goals beyond just beating the game. Though honestly an AI even just beating a random Minecraft seed on normal mode would stun me.
•
u/CriticalDiscipline4 21h ago
This is a great place for them to move the goalpost because LLMs are terrible at real time interaction in the milliseconds.
•
u/cataclysm_catalysis 20h ago
Well they are testing the capabilities and having a way to measure such things is a way to know how the models are improving
•
u/willitexplode 18h ago
Have y’all seen the ping pong AI robot? That thing reacts in ms. Surely it’s on the way.
•
u/DesperateCaterpillar 21h ago
I saw a 5 year old beat Mario Odyssey. It’d be impressive if it beat that since the spatial reasoning is so critical for a 3d platformer
•
•
u/headcrabzombie 19h ago
Age of Empires II with a 60 Actions Per Minute cap might be a good test. I know rn the focus is on getting faster real-time responses, but AoE II is notable for top players still dominating when forced to play at artificially low apm
•
u/Silver-Chipmunk7744 AGI 2024 ASI 2030 21h ago
There was an old game called "Castle Danger"
When i tried to get Sol and Claude to make the AI for it, it was horribly bad.
I think the AI itself did not understand the strategy at all.
I did not test Astra yet, but is suspect obscure strategy games are tough for it
•
u/lil_peasant_69 21h ago
What I can't wait for is to play with these models in games. Like I would love to play gears of war and talk to marcus, dom, baird through my mic and strategise with them and stuff. Woww, I'm hyped just imagining it
•
u/The_Architect_032 ♾Hard Takeoff♾ 20h ago
I'd be very interested in seeing Astra try and play any modern fps game.
•
u/Over-Landscape-5892 21h ago edited 21h ago
I highly doubt it could play Rocket League to any capability. Or speedrun Mario 64 competitively. But what really is the point?
•
u/Maleficent_Disk9583 21h ago
"What is the point"
robotic control to interact with the world in real time
•
u/Over-Landscape-5892 21h ago edited 20h ago
I bet we get automated model RSI to take us there before we manage that ourselves to a very high level.
•
21h ago
[deleted]
•
u/CheeryGeoDuck55 20h ago
So confidently wrong, you're a prime example of the Dunning Kruger effect. They obviously mean LLMs playing rocket league, not bots that were made through reinforcement learning. Bots can play all of these games, but actual AI cannot.
→ More replies (2)
•
•
u/RandomMonies 21h ago
I tried to get a harness going for Hearthstone: Battlegrounds. Should be simple enough, but couldn't get it to play fast enough turns to be effective.
•
u/GraceToSentience AGI avoids animal abuse✅ 21h ago
Just make the benchmark physical like behavior1k, or something even harder.
I guarantee you that it won't be saturated anywhere near as fast as arc AGI-3
•
u/wolfy-j 21h ago
Please, you really, like really-really, dont want models to play CoD or any shooter any time soon.
•
u/dervu ▪️AI, AI, Captain! 21h ago
Why?
•
u/SoupOrMan3 These are the end times 21h ago
Don't you have any imagination? Like at all?
•
•
u/dervu ▪️AI, AI, Captain! 21h ago
Playing is not training.
•
u/SoupOrMan3 These are the end times 21h ago
Yes the fuck it is
•
•
•
•
•
u/TyrellCo 21h ago
Personally wondering if all its training on real world geometry makes it unable to adapt to the weird “impossible” geometries of like Monument Valley or Viewfinder. More out of the box thinking than the skills he’s narrowed on
•
•
•
•
u/darkestvice 20h ago
I mean ... do we actually *want* Astra to become good at first person shooters? ;)
•
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 19h ago
Modded(large modpack like GTNH, or Star Technology), is kinda the definition of the exact challenge we want AGI for no?
•
•
•
u/letmebackagain 21h ago
AI cannot play League of Legends, Dota 2 or StarCraft. They need a lot of Action Per Minute to stay competitive and Tokens are too expensive and not fast enough for that.
→ More replies (3)•
u/Defiant_Station_5895 21h ago
Wasn't there an AI called Alpha star a few years back? If I recall correctly they had to cap its APM so it didn't have an unfair advantage Vs humans.
•
u/letmebackagain 20h ago
Ye but that was a specialized AI for that. Not an LLM or Generalized model I think. But I'm assuming, I don't know the architecture.
→ More replies (1)
•
u/piponwa 20h ago
I'm sorry but how could the president of ARC prize be so short-sighted? Has he never heard of Goodhart's law? He's doing the most extreme version of benchmaxxing. Taking a specific model artifact, defining what eval it would fail at and making that the eval. Then OpenAI will just focus on that and get great at it. Shocked Pikachu.
This guy is supposed to think about what intelligence is, regardless of what models can do today. Not define intelligence in contradiction to what exists today. If you need further indication that the Arc prize methodology sucks, there you go.
•
u/IronPheasant 17h ago
I'm not a fan of some of their hijinks in ARC-3 either, but relax eh. Goodhart's law isn't applicable here.
When the benchmark is literally 'be able to perform these 300 different jobs', is that 'benchmaxxxing'? Or is it just making AI?
'Benchmax' is only applicable to tests that aren't meaningful to day to day work. Trivia, particularly.
How do you 'cheat' at pretending to understand navigating around a 3d world? It's just a blob of pixels. You can stick a noise or color filter over it. Anything that isn't a cognitive pipeline that converts vision input into internal geometry usable by other neural networks in the system, isn't going to be able to do it.
A secondary point that might make you happy: This is basically an open surrender that they no longer have anything left. What are they going to do, make San Andreas? Of course not. Just have it play San Andreas.
The same thing applies to simulation. There are simulators out there far beyond their ability to match or exceed.
ARC-AGI ends at #3.
If ARC-AGI 4 is a list of secret video game evals, or crappy godot low effort knockoffs.... Just imagine how much internet bullying they'd get.
•
u/ShelZuuz 20h ago
Dude, the benchmark isn't the game, it's the human on the other side. The game is just an instrument.
•
u/BrennusSokol AI please take my job 20h ago
You seem weirdly angry and biased against this guy/ARC.
•
•
u/1337-Sylens 21h ago
Just load up 5 bots with aimbot and run them at any pro team, they will completely crush.
→ More replies (2)
•
•
•
u/Sextus_Rex 21h ago
Can someone ELI5 how an LLM would play a real time game?
•
u/dbaugh90 20h ago
Perhaps by creating rules and weights for itself and then running itself on a lower model that processes quickly to make actual moves.
I'm not sure you can get around the latency any other way besides waiting for the models to think faster
•
•
•
u/Historical_Nature574 20h ago
League of Legends. Difficult game and a chance to learn from the worst of humanity
•
•
u/TheMCM80 20h ago
Civ 6, but on the hardware or a really average PC from 2017 so that it has to deal with the game crashing all of the time.
When it can win a Civ 6 game after dealing with the endless crashes… AGI accomplished.
•
•
u/Upset_Page_494 20h ago
Managing to send a rocket in Factorio. The Forest. R.E.P.O solo(reach level 10).
•
u/sirtrogdor 20h ago edited 20h ago
I hope they aren't only considering realtime games because of how quickly you need to react. That's not a very interesting test unless you're focusing on robotics or something where hardware limitations matter more.
If it can solve a game at 0.01x speed then it can solve it at 1x with enough hardware.
I hope they are only mentioning realtime games with the assumption that the computer can pause to think (which is how Astra beat Portal), because it would imply an agent that can strategically choose when to pause and think in order to beat the game within a reasonable $1000 budget or something.
As opposed to turn based where it's obvious when you should pause to think.
EDIT: Now that I think about it, since they mention "latency", they hopefully just mean a harness where the state of the game has changed by the time it finishes a thought. I'd rather see people simulate whatever latency we want. So we can first see if Astra can beat a game under $1000 with "infinite" thought speed (AKA pausing the game while thinking) vs settings with higher budget or slower thought speed (up to realtime or even worse, potentially).
•
•
u/FishDeenz 20h ago
Based on nothing except the Portal/Balatro playthroughs, I feel like Astra *could* do well at Civilization VI. The others probably not as well.
•
u/DetectiveFinch 20h ago
Not an expert, but am I correct to assume that such games would need am AI that uses a world model, like some robots do?
•
u/IronPheasant 17h ago
Navigation is important in most games, yeah.
There's a lot of faculties necessary to beat all arbitrary games, to the point the only thing they don't test is motor function.
(It's the part about sports games that I find to be the most lame thing about them. Driving a vehicle is simulated wonderfully in video games, but throwing a ball or the like? It's not close.)
•
u/Responsible-Laugh590 20h ago
The problem with FPS is that how do you make the shooting part fair when it can mover faster than humanly possible and any sound ticks would set it off in a way that’s not competitive.
•
•
•
•
•
•
•
u/Proper_Actuary2907 Spooky Machine Intelligence 2030 19h ago
I wonder if it actually could play some of these on the fancy fast inference chips that have come out, wouldn't 750 TPS be fast enough?
•
u/MeMyself_And_Whateva ▪️AGI within 2028 | ASI within 2031 | e/acc 18h ago
Let it try Elite Dangerous. It's a long game.
•
•
u/chatlah 18h ago edited 18h ago
Path of exile. Not that i think AI wouldn't be able to play it (it obviously will be), i just want to see what kind of builds / strategies an AI would use. I personally would love to see AI capable of making very advanced builds from scratch by analyzing the game data, without cheating by looking at the human created meta.
As for shooters, as a long time fps player i am not excited to see AI play that genre whatsoever. Bots in fps games were a thing for a long time now, and AI playing fps games would just be a more advanced version of bots. At their core fps games are simple actually, a map, weapons, player models, AI will very quickly figure that all out.
I am more interested in AI playing arpgs because those games offer both a challange to control the character and all that but also knowledge based challenges like build making, farming strategies etc. I feel like that is more appropriate challenge for an AI vs simple fps games regardless of how 'advanced' the maps, movement and weapon mechanics are.
•
u/ActAmazing 18h ago
OpenAi started with Dota 2. A generalist playing Dota 2 and beating humans can probably wrap up the AGI debate.
•
•
u/deleafir 17h ago
So we're finally reaching a point where models will be trained to beat actual videogames on a large scale.
Awesome. AI capable of beating almost any game at ~human speed by 2028?
•
u/Responsible_Camp_559 17h ago
Fenris Games (formerly CCP Games), creators of EVE Online are working with Google to have AI learn EVE Online.
EVE Online will be playable by AI in a couple years, I'd imagine.
•
•
•
•
u/Difficult_Result_225 16h ago
This is actually a pretty good way to probe capability imo. Benchmarks get optimized against eventually, but weird games 😅 can expose whether the model can actually generalize to unfamiliar rules, planning and spatial reasoning. I would be more interested in seeing where Astra systematically fails than another leaderboard score tbh.
•
•
u/jseah 15h ago
How about an obscure game that it hasn't seen before and the UI is scrambled symbols?
I played a touhou shop simulator x tower defence (yes, very wtf) without translation before and figured out the entire UI through sheer trial and error. Learning how to play was more fun than actually playing it.
•
u/Efficient_Loss_9928 14h ago
Anything that requires fast reaction + a human / AI opponent that does random movements.
I would be suprised if you can beat Google's 2019 AlphaStar with GPT-6's standard inference API, the reaction speed is just not there.
•
•
u/trolledwolf AGI late 2026 - ASI late 2027 12h ago
I honestly believe a model could, potentially, be able to play Dark Souls 1. It's a slower paced game and if you allowed the AI to use the wiki to figure out a way to get through each encounter, it might be possible, I'd be curious to see it.
•
u/FireFearing 12h ago
once it can beat challengers in league of legends without game-specific training its 100% agi
•
•
•
•
•
u/Ok_Train2449 7h ago
Would love to see it play old Xcoms, especially to see how it treats the soldiers. And those games require both long term planing on the geoscape and short term tactical decisions on the battlescape, both with never having perfect info.
•
u/Jiffyrabbit 7h ago
Dota 2 / league would be a good one. Particularly if it had to play with randoms.
Would love to see a model respond to being called " a fucking noob" and greifed by someone
•
•
u/funky2002 3h ago
This seems silly, but 100% a creative writing benchmark. Many exist, but they're all deeply flawed. And no, it's not just that they overuse rhetorical patterns like "It's not X but Y". All "slop patterns" are part of a much bigger issue. I rarely see anyone addressing these outside of petty complaints.
Believe me when I say that if you decide to write with LLMs, you will no longer believe them to be AGI. Even the latest Astra and Fable struggle with it. They seem to struggle with intersubjectivity, spatial awareness, and temporal awareness. No amount of scaffolding seems to help. This goes for both formal passive writing and narrative fiction and non-fiction.
•
u/FriendlyJewThrowaway 2h ago
Let's see Astra play Theme Park! Demis Hassabis was one of the lead programmers and designers on that one.
•
•
u/charmander_cha 21h ago
Agora é só esperar uma IA com um cachorro especialista em counter strike me matar no conforto de minha casa
•
•
u/Proud_Ask_9030 21h ago
First Ai to get a fire cape in runescape is AGI.
Nothing less will suffice.