Transcript
00:00:00you're using cloud code in obsidian the wrong way or at least you're barely scratching the surface
00:00:05we can use obsidian to do a lot more than just track markdown files for us instead we can use
00:00:11it to create a command center for our entire cloud os a system that includes a fleet of skills and
00:00:16automations a memory layer and a visual wrapper on top of it all that gives us insights that you
00:00:22just can't get with the terminal or the desktop app did i also mention that we can give it fully
00:00:27local voice capabilities but how do we do this in a way that actually provides value and doesn't
00:00:31devolve into a zero substance visual spectacle well that's exactly what we're going to cover today and
00:00:37by the end of this video you're going to understand how to build this yourself now if you follow this
00:00:42channel for some time you know that we've gone through a few iterations of this clod os not too
00:00:47long ago it looked something like this which was a web-based version and before that we had an obsidian
00:00:54version now the cool thing with this web-based version we had was the voice capabilities a fully
00:00:59local voice setup that allowed us to get very quick responses and control some things through our
00:01:03microphone now what i've done is we've essentially upgraded the obsidian command center and we brought
00:01:10in the best parts of the web version and combined it into what you see here this gives us all the power
00:01:16of obsidian i've included a voice setup which was very much inspired by what we see from codex and chat
00:01:23gpt's voice mode and we now have a one-stop shop for everything we want to do with cloud code but what
00:01:29does this whole thing actually buy us that we don't get inside the terminal don't get inside the desktop
00:01:32app we have to answer that question or else this whole thing is kind of pointless let's start on the
00:01:37visual side with what we're actually looking at first things first it gives us a simple glance at
00:01:41whatever single metric we want what i really want to hammer in here is that this is 100 customizable
00:01:46so whatever you want to be able to see you can put inside here this is just a custom obsidian plugin
00:01:51that cloud code actually built for me i want some sort of insight on what i'm doing with my token usage
00:01:56i can see all of my metrics across social media channels i can see my schedule that is pulled from my
00:02:01google calendar from that it breaks down specific tasks for me and then i also get some insight in terms of
00:02:07my morning headlines right here you can see different automations and skills that i can run at a click
00:02:12of the button and i have different tabs here that dive deeper into these different reports that i run
00:02:18daily for example this is my research report this comes from a morning intel automation which gives me
00:02:24all the github trending repos based on the last seven days the last 30 days and then i can also see
00:02:29outlier videos on youtube different content angles and then i get sort of a running look at what's trending on
00:02:35hacker news when i look at my audience tab this is taking a look at how my specific content is doing
00:02:40across youtube what's working what isn't and why and since this is obsidian i'm not limited just to
00:02:45this i can use the terminal inside of obsidian and this is partly why i like this voice modes because i
00:02:51can pull this up at any time and say something like pull up the terminal
00:02:57and cool there it is now this thing does have actual voice stuff so if i say something like hey what was
00:03:03the biggest ai news story for today claude got a model level watermark rolled out globally yesterday
00:03:13invisible survives copy slash paste triggered by the eu ai act deep seek harness also exploded to 109
00:03:23k stars in two days and is dominating github trending so that's kind of cool that i can like ask it a to do
00:03:32commands inside obsidian like i said pull up my terminal or whatever and i can ask it questions
00:03:37about reports i've already run that's why it was relatively quick it didn't have to like take my
00:03:41command go out on the web find that information process it and then respond it's been wired to
00:03:47look at these reports that i've already run again something like my research report and then quickly
00:03:50answer questions about it the other cool thing about voice mode is this works even when i'm not inside
00:03:54obsidian so let's say i'm tabbed out i'm doing some other work elsewhere like for example i'm inside the claw desktop app
00:04:00yet i want to ask it a question about something i essentially want to call on claude to audibly
00:04:05talk to me again very similar to what we do with gpt live mode which you can do inside of codex this is
00:04:10a really good tool so in this case i just key the hotkey and i can ask something like what's the biggest
00:04:16thing on my schedule for today
00:04:23your biggest commitment today is the sponsor deliverable review for warp at 11 a.m then the chase
00:04:30ai plus coaching call at 3 p.m but the whole day is really built around the voice orb demo record block
00:04:37at 9 30 which is the main focus and so like you see there i'm able to ask questions about different
00:04:46things and it can quickly respond now this also the sort of voice mode thing it doesn't just answer
00:04:52questions about you know reports i have going on it can actually call on claude in the background and
00:04:57run a headless version of it so if i was doing something else and then i actually wanted cloud
00:05:02code to go ahead and do something like relatively deep like hey go ahead and start you know planning
00:05:07this project or start you know doing deep research on x y and z it would go ahead and do that it would
00:05:12do it in a headless mode now the voice response wouldn't be as quick because it has to you know
00:05:16actually go and do it and then respond but for these quick tasks that are pulling on information
00:05:21i've already found it's relatively smooth and again this is all local this is a local voice mode
00:05:27running on my gpu and i'll show you kind of how it's set up but overall i think in terms
00:05:31of the visual page that's sort of the value this sort of like wrapper os obsidian setup is bringing
00:05:37you which i think is pretty cool especially the voice mode when it's tapped out because
00:05:40being able to audibly interact with cloud code even when you're tabbed out and stuff in a way that is
00:05:44similar again to that gpt voice mode is something i wanted to bring over because the voice mode inside
00:05:48of like the cloud desktop app just isn't really there so that's what the user interface and voice mode is
00:05:53actually bringing to the table the second thing that's actually providing value in a cloud os setup
00:05:57is that skill and automation backbone you see all these skills right here that have been you know
00:06:03essentially moved into buttons that i can click at any one time or i can have it run it being voice
00:06:07mode and that's great the real value is that i have taken everything i do in my day to day and i have
00:06:12turned them into skills and automations which is a simple enough concept but very few people do it
00:06:16correctly what you're looking at here is a visualization of all the skills and automations
00:06:21i run and the concept is relatively simple you have a number of tasks that you run on a day-to-day and
00:06:28week-to-week basis both personally and with your job these tasks are broken out into different domains
00:06:34for me that includes things like memory productivity research content my community my agency sales so on and
00:06:41so forth within each of these domains there's specific things that i do for research for example i have a
00:06:48pipeline dedicated to youtube i deep research specific topics i have stuff that goes into my light rag
00:06:53database i have my morning trend scan which is essentially that report you saw earlier i want to
00:06:57be able to look at my competitors etc etc etc i could do all these manually or after i figured out how i
00:07:04actually want it to work manually i turn them into skills and then once they've been turned into skills i ask
00:07:09myself does it make sense for this to be an automation if yes that's what i do now doing this has nothing
00:07:14to really do with like a cloud os in the obsidian dashboard you said this is just like best practice
00:07:18every everyone should be doing however building this is what essentially is the foundation of the
00:07:26cloud os if you don't have this creating something like this is kind of pointless like what are we even
00:07:31doing here there's no like information that's going to travel here you're not going to be able to call
00:07:34on specific skills and automations regularly this becomes a visual spectacle
00:07:39if we don't have this foundation so this is the actual substance and we'll go into the best
00:07:44practices of creating something like this for yourself because it's not really hard
00:07:47and then last but not least we have sort of the obsidian memory layer which i think people get
00:07:52confused with the whole point of the obsidian memory layer isn't that it's necessarily super
00:07:57charging cloud code like we see all the stuff and we get like super hyped up about oh my gosh look at
00:08:02like this graph this looks super cool it's like a graph rag like it's not like a graph rag it's not like
00:08:06graph rag at all actually what obsidian does is it organizes information and markdown files for you
00:08:12the human and there is a small tangential benefit for cloud code itself that tangential benefit for
00:08:19cloud code comes into effect when we have thousands of files and we have set up our obsidian ball in a
00:08:25way that actually makes sense it makes it easy for cloud code to navigate to give us accurate answers
00:08:29quickly essentially saves us on tokens and boosts the performance a little bit and that is entirely a result
00:08:34of how you set it up what you see here is that carpathy obsidian rag system you see everywhere and this is sort of the
00:08:40template for it and again we'll go into deeper detail of how you would set it up because you don't have to do it
00:08:45exactly like carpathy did but there are some fundamentals we can pull from it so that's what the
00:08:50subsidian plus cloud os does that's what it buys you and what you can't get inside the
00:08:54terminal or inside the desktop app and now let's talk about how you would actually create this for
00:09:00yourself but first a quick word from today's sponsor me so i just released a brand new version of my
00:09:06cloud code master class and is the number one way to master this tool especially if you don't come from
00:09:11a technical background we focus on real practical use cases so you can actually learn the fundamentals
00:09:17and then turn around and apply this to whatever personal project you want we're currently running
00:09:22a sale on the membership so if you want to join check the link in the pinned comment you can find
00:09:26it inside of chase ai plus now before i explain how to create your own skill and automation backbone
00:09:31for your cloud os let me walk you through the voice setup so you sort of understand how it's actually
00:09:37working so in this example you are saying hey what's today's biggest ai news this voice command
00:09:46goes to the voice module call jarvis whatever you want and that voice recording is then transcribed
00:09:53via faster whisper which is a open source repo it's going to live on your computer does a transcription
00:09:59from there your voice got turned into words and those words get sent to haiku 4.5 i used haiku
00:10:06because it's the smallest cheapest fastest model we also have the ability if you want to swap out haiku
00:10:12for any sort of local model the reason i don't have the local setup is because
00:10:16i give this exact setup away to people you can find inside of chase ai plus and i don't know
00:10:21necessarily what their hardware is going to be so the power of their local model can change a lot so
00:10:26haiku is like a nice standard but if you have the hardware for it you could change this for really
00:10:30whatever you want throw in quen throw whatever whatever so what's the biggest ai news it becomes text
00:10:35that text goes to haiku and haiku now figures out where to route this and there's really three tiers
00:10:40this is broken into so tier one is skills if i told you know jarvis hey i want you to run the morning
00:10:47intel report well that's going to get routed to tier one and it knows it's just going to have to execute
00:10:51a skill it doesn't have to add any other words to it just execute the skill just do what you're told
00:10:55right and then let me know when it's done tier two is hey i just want you to pull metrics so that's
00:11:01what we're asking here i'm saying what's today's biggest ai news i wanted to just look at a report that
00:11:05already exists so if i'm asking about news it knows it can go to essentially the research tab take a
00:11:11look at our morning intel report and let me know these things we break out this specific tier because
00:11:15we want this to be fast i don't want it to do any like searching or research on its own just find
00:11:19whatever exists and tell me so that would be what this command goes goes into tier two then we have
00:11:24tier three and i alluded to this earlier if i pull up voice mode and i say hey i want you to go ahead
00:11:28pull up fable 5 i want you to start doing deep research on task x y and z and then coming up
00:11:32with a plan for action well that isn't a specific skill that's not a metric we need to actually like
00:11:38pull up an actual instance of clawed code and that's what this is going to do it's going to do
00:11:42it completely headlessly it's going to run all the things and then it's going to report when it's
00:11:45done so this becomes kind of the catch-all this requires the most power and obviously it's going
00:11:49to be the slowest now once it executes any of these tasks whether it's in tier one two or three
00:11:54it's obviously the response is going to be in text but we want that to be a voice response
00:11:59so that is where kokoro comes in again an open source tool which then gets sent back up to here
00:12:06and then it responds to us with you know whatever it's going to say so that's how the voice mode works
00:12:11you say command it gets transcribed it gets routed command is executed turn into a voice sent back up
00:12:18and it's actually simple enough and like i said you could turn this 100 local if you wanted to
00:12:22by simply swapping out haiku for your local model of choice now let's talk about the skills i have
00:12:28some of my most commonly used ones right here so at any time i want to run them and i have this tab
00:12:32open i can just click on them and like you saw as we went through the voice mode we can easily trigger
00:12:37them just by saying the name through the voice system don't even have to be tabbed in but the
00:12:42question becomes how do we actually create these skills and what should those skills look like luckily
00:12:46it's complicated this looks the workflow for doing this actually pretty simple step one is defining
00:12:52the tasks what is it you actually do in your day to day that we can codify remember one of the issues
00:12:59with ai is that it is inherently non-deterministic i tell it to do one thing and i tell through that one
00:13:05thing 10 different times it's going to go about it 10 different ways if i can instead specify the way i
00:13:11want things done and specify an end state we can then turn that into a skill and essentially codify
00:13:16it make it as deterministic as possible but we have to know what those tasks even are so the way we
00:13:21can do that is well luckily we actually have a few options to do this the first option is we just kind
00:13:27of give it a stream of consciousness all right you're going to turn on your microphone and you are just
00:13:33going to yap the cloud code and explain what you do every day and what you do every week based on that
00:13:39conversation we're then going to ask cloud code hey i just gave you a bunch of information about what i
00:13:44do which parts of that can we turn into skills and then it turns them into skills for you the second
00:13:50way to do this is you have it take a look at your actual logs everything you do inside of cloud code
00:13:55every single tool call every single command every single text is logged on your computer we can have
00:14:01cloud code take a look at these logs over the last 30 16 90 days and then pull skills out of that this
00:14:07is really nice because we have an idea in our mind right about what it is we actually do
00:14:13but what is the reality of that right what do you literally use cloud code for we're going to be able
00:14:18to see this in the logs and ideally you would actually do step number three which is a combo of both of
00:14:22these things right you do the screen of consciousness you tell cloud code what you think it is you actually
00:14:27do and it creates skills from that then we take a look at the logs themselves and pull that out and
00:14:33in combination with those two things you begin to build this repertoire of skills now which skills
00:14:40do we turn into automations oftentimes that's going to be pretty self-evident right is this something you
00:14:46need to run every single day to specific time well then turn it into an automation turn it into routine
00:14:51which is one command away you just tell cloud code to do it for you now when it comes to automations and
00:14:56skills really the best practice is to keep it as a skill first for a while actually make sure this is
00:15:01working the way in the manner you want and then once you're happy with the outputs you're getting
00:15:05with the skills then we go ahead and turn it into a full-blown automation that we don't run manually
00:15:10it just does its thing and we can go even deeper with this and i'm not going to do that in this
00:15:15lesson and essentially start adding some like loop engineering principles where we're taking a look
00:15:19at these automations and that run we compare them with their past outputs and we build some sort of
00:15:25system where it becomes self-improving right we give it some sort of specific goal some sort of
00:15:29criteria trying to hit and if that automation doesn't hit it we can continue to adjust but that's
00:15:34a little more advanced just understand that's where you can go with this but it's all built upon the
00:15:39skills right can we codify what you do every day into a skill the answer is yes and it is simple from
00:15:46there integrating that into your cloud os is really just as simple as this like which ones do you use
00:15:50daily and in terms of the voice mode section it already knows what skills exist so there's nothing you
00:15:56really even have to do there remember like it's haiku it's clawed it has a list of everything you do
00:16:00in the background and those skills are also going to form the basis of what your command center even
00:16:04looks like here right everything you see here is essentially some sort of skill or some sort of
00:16:08automation whether it's pulling my schedule from google whether it's the morning headlines whether
00:16:13it's pulling metrics from social media whether it's this research section this is all just outputs and
00:16:18data from automations that's all this is now yours is going to look different and that's a good thing that's why
00:16:24i think the system is great because it is fully customizable whatever it is you want to see
00:16:30should be an output of a skill and then we throw it on here and boom it's yours now let's talk about
00:16:36the memory side of the equation and what obsidian actually brings to the table in terms of quote unquote
00:16:40improving clod's memory now this is the tweet from carpathy that made everyone lose their minds
00:16:45over obsidian and sort of that clod code connection like this was out in april nearly 22 million views
00:16:51and sort of the system he laid out is relatively simple and this is what it looks like our obsidian
00:16:56command center our clod os primarily lives in a folder known as the vault doesn't literally
00:17:02have to be called the vault but that's sort of the naming conventions that obsidian uses it's just a
00:17:06folder on your computer within that folder we have three subfolders we have raw we have wiki and we have output
00:17:15so i want you to imagine we have claude open up in this vault folder and we say i want you to do some
00:17:23research on rag systems so it goes out and it finds a bunch of data a bunch of sources a bunch of raw
00:17:31information about ai asians all that raw data is essentially put inside the raw folder it lives here
00:17:37right we have article about rag now let's say you want to go a step further and you want to take all
00:17:44that raw information you want to turn it into like a wikipedia style article well that thing goes into
00:17:49the wiki section and you can see here we have a sub folder in the wiki section for rag systems so all
00:17:56that data gets turned into a new article about vector databases and we have a second article that has to
00:18:02do with chunking strategies right we're essentially just synthesizing all that raw information we found
00:18:07now let's say we want to turn that into a slide deck i want a slide deck talking about rag systems
00:18:12well then that slide deck is going to get put into the output folder right we're turning into an actual
00:18:17deliverable and it is now a slide deck so the simple idea is we have a section for raw information
00:18:24we have a section for sort of just like synthesized data and then we have the deliverables we create
00:18:28from the synthesized data now the important thing here isn't that we have that sort of you know
00:18:34breakdown it's that it gives ai a clear path forward to find information and the way it does it isn't
00:18:41just because we have the folders it's because at every step of the way we have essentially a master index
00:18:48or an index.markdown file which acts as a table of contents that says hey once you go into the wiki
00:18:53folder you can now read this markdown file that says here's the three subfolders inside here we
00:18:59have folders for ai agents rag systems and content creation does ai need a master index a table of
00:19:04contents when you have three subfolders absolutely not but what if you have three thousand what if you
00:19:08have thirty thousand what if you have three million right it becomes a little bit easy for everyone
00:19:12involved if there is a table of contents and that idea is repeated as we go one layer deep so
00:19:18i went to wiki i saw the master index i wanted to learn about rag systems will i go into the side
00:19:23the rag systems what do i want to learn about rag systems well guess what there's another index file
00:19:27in here that says hey vector databases chunking strategies etc and the whole point of this is
00:19:33is there's really two points one for you if you the human being wants to be able to find stuff it's
00:19:38pretty easy two there's a clear path we give cloud code a map to find things and because it has that map
00:19:44it gives us answers quicker and it's going to burn less tokens doing so and so that's the whole purpose
00:19:48of the obsidian memory it's not really giving it more memory it's giving it a map now in terms of
00:19:52practical application do you have to set up exactly this way with raw wiki and output absolutely not you
00:19:57do not but you need to set up something that makes sense for you you can see here over on the left i have
00:20:03a folder for content daily notes inbox ops project systems and wiki so it is a little bit different and
00:20:09slightly more convoluted if you make something that is slightly more convoluted how do we
00:20:14still get the benefits of a quote-unquote map well you can just spell it out to claude and that's
00:20:18what i do inside my claude.md the biggest thing you're going to do inside of your claude.md that
00:20:22lives in your vault is just spell out the vault structure and how it needs to navigate things because
00:20:27with a navigation pattern and a vault structure i've given it a map and instructions of how to read it
00:20:32that's all you got to do and guess what if you're confused with all that you know what you can do
00:20:36you can literally just point claude at your vault you can copy this entire tweet from carpathy and you
00:20:41can say hey can you set this up in accordance with the principles spelled out here now let's talk about
00:20:45how you bring this all together and actually build this sort of ui again completely custom and it is
00:20:49literally just a plugin that claude code creates itself and installs into obsidian on its own and you
00:20:54can see that here under community plugins i have the chase command center of note one other community
00:21:00plugin you definitely want to have enabled when you're doing all this is the hot reload community plugin
00:21:05you're not going to be able to find it at least you haven't been able to find it through the official
00:21:09obsidian community plugin sort of like user interface but you can find it on github just
00:21:14search for hot reload and it's right here just point cloud code at this and it'll install it into obsidian
00:21:19for you so all you have to do is have a conversation with cloud code and say you want to build a
00:21:23custom obsidian plugin that acts as a command center and then you're just going to spell out what you
00:21:29want it to actually look like a great way to do this is to actually use claw design and have it come up
00:21:35with a bunch of different visual mock-ups for you you can see here how many different iterations
00:21:40i sort of went through so i would give it some inspiration like you can literally just go on
00:21:46something like pinterest and look up like you know dashboard or something like command center to get you
00:21:52started drop in those screenshots in here and then say hey i'm trying to come up with a mock-up
00:21:57for obsidian plugin i want it to bring in these certain metrics that certain metrics will depend
00:22:02on what you want to actually see i want it to show my calendar i want to show these skills i want to
00:22:07show it my token burn whatever it is you want and then you're going to say i want you to create five
00:22:12very distinct variations of this and if you do that inside of claw design it will put them all on the
00:22:17same page like this all of them are probably going to be relatively different and then when you find one
00:22:22you like i suggest you then double down on it and have it create multiple variations of that command
00:22:28center so i kind of went like with this clay hardware one at first then i found this cool like background
00:22:33i was like okay what if we put a background inside of here and then had it layered on top so then i started
00:22:38doing variations of that eventually i landed on this one where we kind of have that glass morphism effect
00:22:43so you can kind of see the clouds sticking in through the back here and then if you look closely i had
00:22:49it also add almost like this like star twinkle effect it's pretty subtle so because i didn't want
00:22:55to be overwhelming um but that's what i did i added a little bit of motion so when i click the overview
00:23:00you can see it tick up but again like you can do whatever you want you make this as cool or as lame
00:23:06or as boring as you want to see totally up to you but if you're doing this inside of claw design once
00:23:12you land on a sort of design you want to move forward with you just tell it that and it's going
00:23:17to take all the code it's created for these mockups it's going to put it in a zip file and then you can
00:23:22just take that zip file like you see right here drop it inside of claw code and say hey build the plugin
00:23:28based on this it's going to build it it will install it and then you'll be able to see it inside of
00:23:32obsidian itself if you want to go a step further you can actually enable computer use inside of
00:23:37clawed code and you can have it sort of do all the design iterations while it's taking screenshots
00:23:42of itself so it kind of makes it pretty hands off but that's essentially the process i went through
00:23:47to create mine and so that's the workflow for creating your own obsidian clawed os command
00:23:51center with some voice capability like i said you need to customize this to get the most out of it
00:23:56but if you want my exact setup including the whole voice thing and everything i have that inside
00:24:01of chase a plus and you can find that in the pink comment so as always let me know what you thought
00:24:06i would love to know what sort of command center sort of setups you've been able to create and what you
00:24:11did to customize yours because you can get kind of crazy with this but besides that i'll see you around
Community Posts
No posts yet. Be the first to write about this video!
Write about this video