Fable 5 Rules Anthropic Doesn’t Want You to know
AAI LABS
Computing/SoftwareManagementInternet Technology
Transcript
00:00:00you already know that soon enough fable 5 is going to be taken off our plans but you shouldn't be
00:00:04worried about that anthropic already posted that they plan to bring it back so it's certainly not
00:00:09going away forever now when fable came out anthropic put out a prompting guide and you
00:00:13might think that those are the only rules for using the model but they're not that guide is
00:00:17just for using the model in an app it doesn't tell you how to actually use it in your own workflow
00:00:22you already know that we're a software company and due to that we've been running this model for
00:00:27days now both in our own products and in the ones that we've been building for our clients and from
00:00:32that we've gathered certain rules that you can implement for example what settings should you
00:00:36actually use to get the most out of the model then there's a free tool that you can use to reduce your
00:00:41fable usage but in order to use it you need to make sure that your coding project is set up in the right
00:00:46way we do know that we're a bit late but by the end of this video you'll clearly understand how you need
00:00:51to start using this model so the first thing you need to change is the effort setting and this one
00:00:56alone will cut your usage in half without decreasing the quality at all now you already know effort is
00:01:02just how hard the model thinks before it answers you but what you probably haven't thought about
00:01:06is that on fable 5 turning it up is actually a huge trap here's what you probably do you see the max in
00:01:11the menu and you think that it's the smartest model so it should work as hard as it can and you crank
00:01:16it all the way up but this changes nothing for the output that the model gives you it gives out the
00:01:20same output on x high and max as it does on high this was honestly surprising to us and it's not
00:01:25just us other people have experienced this as well if you push it to higher settings it's just going
00:01:30to get brutal on your usage and ultra code is just a clawed code skill that has been attached on top of
00:01:35the effort level which we honestly think hasn't really produced anything worthwhile ever it's just
00:01:40one of the ways that anthropic gets you to spend more on their product so keep the effort between low and
00:01:45high and never go past it and honestly you'd be surprised at how well the low and medium effort
00:01:50levels perform on fable now what happens whenever a huge model and a smaller but smarter model come
00:01:55out one of the most common patterns that you see throughout the ai space is that people start
00:01:59saying you should use the larger model for planning and the smaller model for carrying out the actual
00:02:04work and this happens every single time when sonnet 5 came out the new trending lesson became that you
00:02:10should use sonnet 5 for all the smaller tasks that are not worth the opus model because
00:02:15fable hadn't come out at that time still it's not bad advice for example if you've built a second
00:02:20brain you already know that if you want to ingest documents or read huge pdfs you shouldn't be using
00:02:25large models for that this is what we do in the ai lab second brain when we want to ingest competitor
00:02:30content for research and even though you might have heard it a lot of times we've still found some
00:02:35tools that might help you out in using multiple models in your workflow if we talk about the workflows
00:02:40that you can use then there's the most simple one in which you plan out with one larger model
00:02:44and then ask the smaller model to implement it for example in our community project you can see that
00:02:49we have the docs folder and inside that we have a subfolder which we always use while implementing a
00:02:54new feature and whenever we come up with a new feature it always has a prd and this prd is based
00:03:00on this template whenever we create a new prd we use fable or other larger models and implement it with
00:03:05smaller models if the new feature isn't too big now you might be thinking can you copy the same template as
00:03:11us you can't really copy it because it's specifically tailored to this project where whenever we make a
00:03:16new feature it always asks how it's going to affect different parts of the project for example are there
00:03:21going to be any new users created in the project and then the data model and migration means do we
00:03:26actually need to change the database behind the app or not also this custom community is where you'll find
00:03:31all the skills design systems and things that we build and show you inside our videos so if you found
00:03:37value in our videos and want to support the channel this is the best way to do it the link is going
00:03:42to be in the description there's another important part of working with agents which is spinning up
00:03:47multiple sub-agents and for that theo here actually gave a relevant section of his claude.md which makes
00:03:53a lot of sense for these sub-agents he has said that for running a task via sub-agents the main
00:03:58model orchestrating the session should have a model graph like this where they should be rated according
00:04:03to their cost intelligence and taste although we haven't really implemented this in our own
00:04:07workflows using the claude.md but we do manually route through these different models and the
00:04:12ranking that theo has done is pretty much accurate here so if the main session wants to use claude
00:04:18models it can obviously spin up sub-agents using that model but how does it use the codex models theo
00:04:24has written that the codex models are only reachable through the cli but for this specific
00:04:28purpose open ai has actually made a plugin for claude code which you can install with this
00:04:33claude code slash command and although it does require having the codex cli it does give you a
00:04:39handful of commands that you can go through if you like these codex models we ourselves have started
00:04:44using codex a lot as well but we use them for review work because we think that the codex models are way
00:04:49better at reviewing what other models have done but we haven't really compared them to fable 5 yet
00:04:54now even though we just told you that you should be using the larger models for planning there's a
00:04:59particular thing we've seen while working with fable 5 this model is way better at finding creative
00:05:04solutions to hard problems so occasionally we've also enjoyed using it directly in our coding tasks
00:05:09as well but that still doesn't solve the original problem which is that it costs too much to solve
00:05:14this you can actually use ponytail which is a set of rules that basically makes your ai coding agent
00:05:20write less code and it's not doing anything new it's just a list of old coding rules that makes your
00:05:25models write less code but if you just go ahead and use it to write less code chances are that your
00:05:30app which already has users might break so you need a certain structure inside your app so that you
00:05:35can start using this tool now you might already know the concept of tdd which means test driven
00:05:40development where tests are written first and then the agent writes code as you already saw we make prds
00:05:46for every new feature and the tests are written based on those prds but before we move to how we
00:05:51use it let's have a word by our sponsor veed if you've ever tried creating videos with ai you know
00:05:56the biggest problem is that every model lives on a different platform and you end up juggling five
00:06:01subscriptions just to test what works veed is an all-in-one ai-powered video creation platform that puts
00:06:07every major ai video generator in one place their gen ai studio lets you type a simple text prompt and
00:06:13get a full social media video instantly script visuals voiceover subtitles all done they even
00:06:18have their own model called veed fabric 1.0 that turns a single image into a realistic talking video
00:06:25up to five minutes long and if you want to compare different ai video generators side by side their ai
00:06:30playground gives you access to google vo cling c dance and more without leaving the platform plus
00:06:35there's a full video editor with dynamic subtitles background removal clean audio and auto resizing built in
00:06:42sign up with our code in the description for 30 off on your first month for example we have the
00:06:47community dashboard here and if i go back you can see that alongside the files having the dashboard
00:06:52code we also have tests for the dashboard as well so whenever anything is changed in the app we run
00:06:57these tests these tests make sure that whatever the dashboard is supposed to do it keeps doing
00:07:02that for example if i click on the settings icon it is supposed to open up this menu a test file tests
00:07:08that this still happens if the agent writes some code that breaks the button and doesn't show you the
00:07:13menu that means the original code has been broken so this is why these tests which are called unit tests
00:07:18exist because they test a single unit which in this case is the dashboard so just like the dashboard the
00:07:24whole app has tests written for it so whenever we are going to make a new feature new tests are written
00:07:29first and those tests are not written by the same agent those tests are written by the tdd test author now it is
00:07:35important that another sub-agent writes the tests because the existing agent will often write tests
00:07:40that are supposed to pass its own faulty code so because our repo has been structured in this way
00:07:45and we always write tests first no matter how the model writes code we always know that it will work
00:07:50also if you want to use this tdd test author agent in your own projects you can get it inside our ai
00:07:56labs pro community as well since ponytail is a plugin in claude code it comes with skills and other stuff as
00:08:02well but you only need the prompt in its skill.md and then you can put that in your main claude.md or
00:08:08paste it as a prompt before you start using fable 5 for coding now you might be thinking how do fable
00:08:135 and ponytail come together so if you are not familiar with the concept of refactoring it basically means
00:08:18that you modify the code so that there is less of it and ponytail is really good in refactoring code
00:08:23since we actually have these tests written whatever it writes using ponytail i always trust that it will work
00:08:29because those tests are passing and in this way you can build apps and constantly keep changing code
00:08:34before we continue to the other rules there's one thing we wanted to clear up there's this one
00:08:38benchmark going around that basically says that fable got dumber and that it can't really code anymore
00:08:44because it will route to opus when it is asked to code and honestly it's just not true it's only this
00:08:49one benchmark and we honestly don't trust it because it ranks sonnet 5 as the number one model in
00:08:54reasoning which is just not true we've been using this model all this time and we haven't seen any
00:08:59rerouting encoding tasks other than when we ask it to explain its reasoning so you shouldn't really
00:09:04get paranoid about this at all we've also seen what other people have been doing with this model and one
00:09:10of the most important things they've forgotten is the reason anthropic released it in the first place
00:09:15they said mythos was too powerful and based on the testing it could find security risks that no other
00:09:20model had which means it could jailbreak those as well so take out your code review agents and your
00:09:25security review agents and specifically focus on security so for example we're currently working
00:09:31on adding new features in the community and a major redesign but in here we basically use some sub
00:09:36agents as part of our verification loop while we're actually building the platform so for example one of
00:09:41them is the code review expert which is basically a sub agent built on top of the skill from cursor
00:09:46called the thermonuclear code quality review and it's a really great skill that actually checks the
00:09:51architecture of the code and make sure that the code base is going in the right direction when we're
00:09:56constantly adding these new features using our agents and you can see that it went ahead and came
00:10:01back with a small change that we need to implement in the app and two other minor ones i'm not going to
00:10:06show these changes to you but we have patched them in and then another sub agent that we run is the
00:10:10security reviewer sub agent which is actually based on this github action called the clawed code security
00:10:16review a github action is an autonomous review that runs when you upload your code to github and you can
00:10:22actually give the link of this repo to clawed code and just ask it to convert it into an agent and it's
00:10:27going to do that now this was a security review of the new feature that we added and this time it said
00:10:32that it came back clean and that there was nothing that needed to be changed but when we actually ran it
00:10:37on the whole platform that was set up it spun up six of these security reviewer agents all based on
00:10:42fable 5 and they were running in parallel across the whole platform which was divided into multiple
00:10:47parts and this proved much more fruitful it came out with a lot of bugs on the platform which we have
00:10:52now successfully patched so you should pretty much focus on the security and code base reviews which is
00:10:57what anthropic had actually warned and advertised it for so recently tarik who is from anthropic also
00:11:03published a field guide to fable in which he described his own experience while working with fable 5 and
00:11:09what he thinks about it in my opinion you only need to focus on some of the things here he has actually
00:11:14made some example artifacts which explain things much better than the article in the artifact he has
00:11:19divided his process into multiple steps i think that you are already implementing these processes with
00:11:25the old models so you do not really need to dive deep into it but one thing which you should start
00:11:29doing if you are not already doing it is first making mockups of the features that you are going
00:11:34to implement so if we look inside our repo again you can see that we have this design folder and inside
00:11:39that design folder we have a mocks folder now whenever we implement a new feature that feature is
00:11:44first added on to this html mock-up this html mock-up is a one-to-one prototype clone of the whole
00:11:50website and we basically add the new feature on top of this before we actually start to work because
00:11:55it is important that you visualize your changes but other than that one thing that i liked in this
00:12:00article was quizzing before merging so you already know that while working on coding you basically
00:12:05create a git branch make the changes in that and then you merge now this entirely depends on the
00:12:10person working on the ai agents but some people do not really look at what the agent has implemented
00:12:15and i am not saying reading every line of the code but just reading a small summary of how the agent did
00:12:20what it did and that is actually really important so the prompt that he has given make sure that before
00:12:25you merge fable actually comes to you and quizzes you on these changes and this practice basically makes
00:12:30sure that you know what is going on in your project and i highly recommend this over time we have worked
00:12:35with a lot of projects including our custom community and custom automations to make our own process
00:12:40better and multiple other apps for clients as well and we have seen that if you do not really have
00:12:45context on what is happening in your code base you will not be able to steer your agent no matter what
00:12:50you do whether you use a goal or you use a loop it will have to end and you will have to add new
00:12:56features or make changes to your project and not knowing what is in your project will come back
00:13:00to bite you at that time that brings us to the end of this video if you'd like to support the channel
00:13:05and help us keep making videos like this you can do so by using the super thanks button below as always
00:13:11Thank you for watching and I'll see you in the next one.