Give me 9 minutes to show you why you really should not sleep on Jev

EnglishالعربيةDeutschEspañolFrançaisहिन्दीBahasa Indonesia한국어PortuguêsРусский中文
MMaximilian Schwarzmüller
컴퓨터/소프트웨어창업/스타트업AI/미래기술

스크립트

00:00:00a couple of hours ago a new ai model dropped and i genuinely believe that it could be a game changer
00:00:07now i know i know we get new models almost every week but i'm not talking about a new
00:00:13large language model here instead i'm talking about jeff by typesafe ai and i know this sounds
00:00:19like a personal assistant like a grogbot like jeff right and it's a typesafe ai that sounds
00:00:25like a typescript library but none of that is the case it's a new kind of ai model it's not
00:00:31a large language model but it seems to be a pretty smart model and it gives us something large language
00:00:39models only kind of give us now just to get this out of the way right away by the way this is not
00:00:44sponsored or anything like that you can head over to their website typesafe.ai and and there you can
00:00:51join a waitlist and i did that like everybody else yesterday evening in german time and i got access
00:00:56this morning so here i am now what is it though the idea is that you have a model that's not about
00:01:04generating text but that can understand text you can pass in any text you want and then you give it some
00:01:12choices so for example here's a basic example i have some sample texts and that could be some emails or
00:01:21some chat messages by customers right you have like a chat on your website customers can send messages
00:01:28that could be these messages here and then you want to kind of decide what to do with them of course
00:01:34you can funnel them through a large language model a couple of problems here for one prompt injection of
00:01:40course second problem it can be rather expensive because you need a relatively smart model so you
00:01:48you'll be burning a lot of tokens there because you need to pass all your tool descriptions to that model
00:01:54you need to pass some extra information to the model the customer message of course
00:02:00and you'll burn tokens along the way and third it's relatively slow right these frontier large language
00:02:06models they take some time to answer jeff is indeed super fast we're two we're talking milliseconds
00:02:12here it's not running on your machine it's still in the cloud but it's super fast and the idea is that
00:02:18you give jeff the message so here ask jeff is really just me using this client and talking to the ai
00:02:26so you give it that message and you can indeed pass in more complex structured data too so it doesn't
00:02:32have to be just a string and then you give it information about what it should do with that
00:02:39message you give it your questions to be precise and there are three types of questions there are
00:02:46choices scoring and now now choices that is exactly what it sounds like you give it some options
00:02:53and then the ai model gives you back some confidence about which choice which option is right
00:02:58score is you let the model score you give it some
00:03:02levels so to say and it creates a score based on those levels and now are yes or no questions
00:03:10so here in this basic example for example i get some user messages i send them to the model
00:03:15and then here i define my questions first question is which team should handle this message and i give
00:03:21it three options and this is entirely up to you so you can really define whatever you need here you
00:03:26can add more options i think up to 10 right now um and you describe what each option is about so i
00:03:33have a billing team and i make it clear that this team is about handling charges invoicing and so on
00:03:38and i got some other teams here in this demo so that's the first question i'm sending to jeff based
00:03:44on that customer message but it's not all because you can have multiple questions in one round
00:03:49we could have multiple choices but here i want to show one of each category so i have one choice
00:03:53question i have one now question i want to ask does this message express urgency and we'll funnel all
00:04:00three messages through it so that we can see for each message if it has urgency and then the third kind
00:04:07of question i'm sending here is a scoring question where i want to know how frustrated the customer
00:04:11appears and if i run this here uh like this we can see this executes and this is really fast so
00:04:20this was sent to the cloud to the ai model and handle and everything but there is no large language
00:04:25model involved instead you can see for this first customer message here the choice was that the
00:04:30technical team should take care of that with a confidence level that's automatically created by jeff
00:04:36that yes or no question whether it's urgent yes it's quite urgent here we have a quite confident
00:04:42score almost one that it is urgent and the frustration is pretty high as we can see second
00:04:49question was um that we were charged twice and we want an immediate refund so that should be funneled
00:04:56to the billing team jeff decided it is rather urgent and the frustrations also it's kind of high and then
00:05:03that third question that's for the sales team it's not urgent and there's no frustration in there now
00:05:09as you can see or at least that is what i see here this could be pretty pretty useful we can build this
00:05:18with large language models but i outlined what their disadvantages are this is fast and it's super
00:05:24cheap we have to talk pricing this costs 42 dollars per billion billion not million like all the other
00:05:33models billion input tokens and output tokens are free output tokens cost nothing so this is vastly
00:05:40cheaper than all these large language models you can send a lot of complex questions to jeff and it'll cost you
00:05:49almost nothing and again the output tokens do cost you nothing and it's super fast and it is purpose
00:05:56built for these decisions and if you think about it and that's why i'm quite excited about this if you
00:06:02think about it a lot of what we're building comes down to decisions every ai agent you build it's full of
00:06:08decisions which tool to call when these are all decisions now as i mentioned you can of course combine it
00:06:15with large language models though here i have another example where i build an assistant and there i got
00:06:22a couple of very simple tool calls here um where let's say this is some home assistant where we can
00:06:29turn off the lights in a certain room or turn them on so that is a dummy function for that a dummy tool for
00:06:35that but where we also may want to use a large language model for certain tasks because again jeff can't
00:06:42draft draft text so whenever you want to generate text for example a reply email for a customer or something
00:06:48like this you need to bring in a large language model and you can do that here i use a large language
00:06:54model and i'm doing that with the ai sdk by versell i'm using gpt 5.6 terra on medium thinking efforts so
00:07:02relatively affordable and then i let jeff make the choices which tool to use so again that's jeff not the
00:07:10large language model but then i execute those tools and in some tools a large language model is used
00:07:15and in some tools it's not and and that's really the amazing part we're building systems where we
00:07:21have to make decisions all the time where we need to score stuff or answer yes and no questions all
00:07:26the time and right now we're often using large language models for them for that and they are
00:07:31good don't get me wrong and one advantage they definitely still have is for example context
00:07:36window the docs mention a budget of around 32 000 tokens of text that you can pass in in your
00:07:45questions here so you have limitations like this in the docs there also is an entire article about the
00:07:50jaggedness of their current frontier model where it's not good at math numbers for example date and time
00:07:56comparison so there are aspects where it's not good now for one i would expect that like all the other ai
00:08:05models it gets better in the future and problems can be solved and besides that again it's not an
00:08:12either or it's it's an addition it can be a nice extra tool in addition to large language models and if it
00:08:18would be equally slow and expensive i wouldn't really see the benefit necessarily but it's not with the
00:08:24advantages it offers it seems really promising now we'll have to do long-term testing see it in
00:08:32production see it on more complex use cases to evaluate if it is very prone to errors if it's maybe very
00:08:40prone to prompt injection or anything like that if it can suffer from problems like this but it looks
00:08:47really useful and unlike for example the entire subquadratic thing that happened a couple of months
00:08:53ago this is something you can use and test it's available right now at least if you get access
00:08:57you can sign up to the waitlist and for me it then was just a couple of hours and it it looks really
00:09:03useful i think we can build really great programs and systems by combining jeff with traditional
00:09:12programming so with normal deterministic code and then also large language models and build really amazing
00:09:20things with all of that so yeah i hope this is a nice first impression overview of what jeff is sign up for
00:09:27the waitlist if you wanna get your hands dirty yourself and then dive into it and play around with it

핵심 요약

TypeSafe AI's Jev offers a sub-second decision framework costing $42 per billion input tokens with free output tokens, replacing expensive LLMs for routing, tool selection, and classification tasks.

하이라이트

  • Jev by TypeSafe AI is a specialized decision-making model operating in milliseconds rather than serving as a standard text-generating LLM.

  • API pricing sits at $42 per billion input tokens with zero charge for output tokens.

  • Input text is constrained to a working context budget of approximately 32,000 tokens.

  • The model handles decisions through three discrete question types: choices (up to 10 options with confidence scores), scoring, and binary now (yes/no) determinations.

  • Current limitations include degraded performance on math, numerical comparisons, and date-time logic.

타임라인

Architecture and Disadvantages of Traditional LLMs

  • Jev operates as a specialized text-understanding model rather than a generative large language model.
  • Routing customer requests through standard LLMs introduces security risks, excessive token consumption, and slow response times.
  • Cloud-based processing yields inference latencies measured in milliseconds.

Standard frontier models burn large amounts of tokens due to extensive prompt boilerplate and tool descriptions required for basic classification. Ingesting raw customer text into full-scale language models exposes application pipelines to prompt injection attacks. High token costs and latency make large models inefficient for high-volume text decision routines.

Question Categories and Automated Support Operations

  • Input processing supports structured data beyond plain text strings.
  • Decision logic relies on three core formats: multi-choice selections, numerical scoring, and binary yes/no flags.
  • Evaluations execute in parallel across multiple questions in a single request cycle.

The system processes incoming customer messages against defined operational parameters. Up to 10 choices can be assigned to a single query, such as routing between technical support, billing, or sales teams based on descriptive text criteria. The model outputs confidence values alongside urgency boolean flags and customer frustration level scores.

Cost Efficiency and Token Economics

  • Input token pricing is set at $42 per billion tokens.
  • Output tokens carry zero cost across all requests.
  • Sub-second execution speeds enable high-frequency decision routing inside software agents.

Traditional LLMs charge high rates for both input and output sequences, scaling exponentially when evaluating continuous decisions. Pricing Jev purely on input volume eliminates output token overhead entirely. This cost profile suits agentic systems that require constant decision points and conditional branching.

LLM Hybrid Integration and Technical Constraints

  • Text generation tasks still require traditional language models like GPT-4 variants.
  • Context capacity is currently limited to a budget of around 32,000 text tokens.
  • Performance degrades when processing math, date comparisons, and numeric operations.

Integrating decision models with SDKs like Vercel AI SDK allows deterministic execution pipelines to switch off light controls or trigger external utilities without invoking full-scale LLM reasoning. The model offloads decision branches, calling text-generating LLMs strictly when drafting customer responses. Current implementation bounds include a 32k token limit and known weaknesses in mathematical logic.

커뮤니티 글

아직 글이 없습니다. 이 영상에 대한 첫 번째 글을 작성해 보세요!

이 영상에 대해 글쓰기