Everybody's building AI slop games ... what about coding though?
MMaximilian Schwarzmüller
Computing/SoftwareVideo & Computer GamesPhotography/Art
Transcript
00:00:00so last week i was on vacation so i only watched the launch of the gpt-6 astra model from afar
00:00:08with some delays since i spent my day robber at the beach then on x or in the internet in general
00:00:15but of course the launch followed the typical pattern we had posts by people hyping up the
00:00:21model people that had early access one could say influencers and then of course a couple of days
00:00:28later after the rest of the world was able to use the model we had more sober or maybe disappointed
00:00:37posts by people that shared that the model seemed overhyped and maybe isn't that amazing now this
00:00:46week i also got my hands on it or i started working with it and where do i start the model is really
00:00:52great at 3d stuff i mean i don't know about you but my timeline has been drowning in blender 3d art and
00:01:02games created by astra now it's hard to verify if everything is truly generated by astra but i got
00:01:08some hints because i also played around with it and at least for the games there are some recurring
00:01:18elements you can spot in these different wiped up games nonetheless clearly the 3d capabilities of astra
00:01:26are really amazing especially compared to what the models were able to do before so clearly it has been
00:01:32heavily fine-tuned and optimized to be better at that kind of work there has been a lot of 3d stuff in the
00:01:40post training data in the fine tuning process i would guess but what about coding that is the part i
00:01:48personally care most apart about and in this launch blog post we of course also have coding related
00:01:57benchmarks and here astra also does really really well now i will say that benchmarks are one thing
00:02:06you know that we had that in all these launches it is possible to optimize models to do well in benchmarks
00:02:13and of course you're heavily incentivized as a lab as an ai lab to make your model do well in benchmarks
00:02:21because imagine releasing a model that does not do well in benchmarks it wouldn't count if it does well
00:02:27in actual usage because nobody would start using it in the first place so i don't blame ai labs those for
00:02:34benchmarks you basically have to do it but benchmarks therefore also don't tell us that much it's it's a nice
00:02:41indicator but yeah you all know it benchmarks are not everything it's more the vibe of using it and
00:02:49and how it behaves when you actually start using it in your day-to-day work and here gpd6 astra for me
00:02:57at least feels a lot like gpd 5.6 sol it maybe has a different set of quirks and features but in the end
00:03:08it's still a good coding model it's a very good coding model it's not horrible at least for me but it's also
00:03:16not groundbreakingly better you still have to know what you're doing and what you're asking for and
00:03:23of course you still can also vibe code stuff that was possible before and that can get you quite far and
00:03:29for many tools or things you needed maybe all you need you don't need to have the perfect code base for
00:03:35every little project or internal tool you need but if you do care about your code base your architecture
00:03:42if you're building something you plan on maintaining and building up one it's the same as with the
00:03:48previous models you need to have proper constraints you need to know what you're doing what you're asking
00:03:54for you need to know how to work with such a model and then you can get good results out of astra that
00:04:01so that has been my experience it still can lead to ugly code it still can lead to unnecessarily complex
00:04:08code though at least in that respect for me it tended a bit less to add layer over layer of
00:04:15complexity and fallback code here and legacy compatible code here which gpt 5.6 sold it a lot
00:04:23it always treated every piece of software as if it were a big well-established enterprise software that
00:04:30needs to be maintained and that existed for 10 years but that's a different story astra is a bit better
00:04:36there for me but it still is a model you need to steer and control and i don't know if that will change
00:04:44when that will change clearly these models are getting better and clearly these models are
00:04:50good at following instructions i mean it's easy to forget this but a year ago we had models that
00:04:57stopped early that didn't do that well when it came to follow instructions that often went astray and
00:05:05whilst that still happens today they're of course much much better they can work on their own on a task
00:05:11for a very long time frame you just want to make sure that you defined the task such and have guidelines
00:05:18in place to ensure that it doesn't wander off but they are much better there and especially of course
00:05:24if you give the model something it can evaluate it can measure it can really dig into a task and keep on
00:05:33going and ultimately solve it maybe not in the most beautiful way but that again doesn't count for every
00:05:40task and depending on your setup it can also solve it of course in in good ways so gpt6 astra is a good
00:05:46model but it's also not the last model we'll ever need and i mean that shouldn't come as a surprise on
00:05:53the other hand i think it's just also the the current meta in which we are where all these labs or especially
00:06:02open ai anthropic in the end they hype up their models so much when they are released and they probably
00:06:09have to to please investors and to get the traction they want to get for these models to keep people in
00:06:16their subscriptions or have new people sign up for subscriptions they have to hype up everything so much
00:06:23that you just you have to be disappointed at some point because nothing can live up to that hype i'm a
00:06:30gamer and you know the same from video games some games are just hyped up that much gta 6 maybe well i
00:06:36think it will be a good game but we have games that are hyped up so much that when they're actually
00:06:41released it's underwhelming or it just doesn't live up to the hype and i think we're constantly seeing
00:06:47that with these models now of course combined with the fact that we get new models almost every week it's
00:06:54really exhausting especially also combined with the fact that we're talking about our our future year
00:07:00we all don't know what we're going to do in in a few years we don't know where our profession is
00:07:07heading and that's not just true for programming it's true for all white collar work we don't know where
00:07:13we're heading we feel like we constantly have to learn new stuff and that of course is exhausting
00:07:18and i feel that's like another big part of why it's so easy to feel really annoyed or disappointed by
00:07:27these model releases because you have that mixture of all these different parts of the exhaustion of
00:07:33the hype and then the the model not living up to it and yeah these are my two cents after being back
00:07:39from vacation it's a good model it's it's much much better at 3d work for coding it's just a good model
00:07:46use it learn how to work with ai properly and i will soon have more to share on that
Community Posts
No posts yet. Be the first to write about this video!
Write about this video