An Introduction to Jev

The IT world is losing its mind over a new type of model released a few days ago.Most AI models we use today are built around one basic idea: give them input and ask them to generate text.Ask ChatGPT a question, and it generates an answer. Give a coding model a specification, and it generates code. Give another model a document, and it generates a summary.This works extremely well when the output is intended for a human. But a lot of software doesn’t need another paragraph of text — it needs a decision.Should this request go to the billing team or technical support?Does this document contain personally identifiable information?How relevant is this piece of text to a search query?Is this transaction suspicious enough to require additional checks?These are quite different problems from simply generating text.That distinction is the idea behind Jev, a new AI model from TypeSafe AI. TypeSafe was founded by Diogo Almeida, who previously worked at OpenAI and co-invented RLHF and InstructGPT, the methods that led to ChatGPT and GPT-4. His research has helped shape modern language models, and he continues to push the frontier of AI systems.Jev is the company’s first System One Model, a type of model designed specifically to make fast, structured decisions that can be consumed directly by software.Rather than returning arbitrary text, Jev returns values from structures that we define beforehand.Or, as TypeSafe describes the idea:“Decisions, not strings.” One of Jev’s most striking features is its cost. Input is a fraction of a cent per million tokens and output tokens are free. The problem with using a "regular" LLM as a decision engineBut hang on, Claude, Codex and other LLMs are supposed to be intelligent; can’t they do that?Yes, they can, but with a few caveats. Suppose we have a support ticket:We want our program to determine which department should receive it.With a conventional LLM, we might write something like:Hopefully the model responds:We can then parse the output and use it in our application. This approach works, and millions of applications already do variations of it.But there is something slightly strange happening. We are using a model capable of generating arbitrary sequences of words just to obtain one value from three possibilities.Because of the inherent randomness built into models, the model could theoretically return:or:...or malformed JSON, additional commentary, an unexpected category or something else entirely.Structured-output systems have made this considerably better, but underneath we are still using a generative model to perform what is effectively a classification operation.Jev approaches the problem differently. The possible output space is defined beforehand. Instead of asking a model to generate the answer "billing," we ask it to choose from a known set of alternatives. In the above scenario, Jev is guaranteed to return either the word billing, technical or sales.System One ModelsTypeSafe calls the model architecture behind Jev a System One Model.The name comes from the System 1/System 2 distinction popularised by psychologist Daniel Kahneman in his book Thinking, Fast and Slow.System 1 describes the sort of rapid judgement humans make without consciously working through a long reasoning process. System 2 is slower and more deliberate.Jev is aimed at the former whilst Claude, Codex, etc. are aimed at the latter. Give Jev some state, ask it a specific question and get a quick judgement back. It is not intended to sit there producing a long chain of reasoning or writing an essay.TypeSafe describes the model as:The difficult part is that some conditions cannot easily be represented using traditional rules.For example:How exactly do we model customer_appears_frustrated? This is where Jev can effectively become what TypeSafe calls a smart if-statement.The model evaluates the fuzzy semantic condition. Our normal code decides what happens next.Three types of questionsJev currently reduces these decisions to three main primitives:ChoiceScoreNoulEach represents a different type of decision.ChoiceChoice is used when we want Jev to distinguish between a predefined set of alternatives.For example:Conceptually we could ask:Jev will never invent a new department or add descriptive words to the answer. It only chooses from the choices we supplied. Importantly, the result can also contain the probability associated with each possible answer rather than simply returning the winning label.We might receive outpout like:Our program can then decide what to do with that uncertainty.For example:ScoreSometimes there isn’t a collection of unrelated categories. Instead we want to know where something lies on a scale. Imagine that we want to measure the urgency of a message.We might define levels such as:That is a Score.The levels have a meaningful order. This could be useful for things such as defining:The important point is that we define what the scale means. Jev then evaluates the input against that definition.NoulThe final primitive has the slightly unusual name Noul. A Noul represents a yes/no proposition and returns the probability that the proposition is true.For example:A result close to:means strong evidence for yes.A value close to:means strong evidence for no.And:means the model is uncertain, i.e. the probability of the proposition being true (or false) is approximately 50%.A practical ExampleSay we want to develop a system that checks input files before processing them. The system should:a) Detect that we are dealing with a text fileb) If we are, determine whether the file contains:This is a good example of where Jev starts to make sense. Although traditional code can easily handle the first part, we’ll use Jev’s choice operator for that as well to show we can use different Jev primitives in the same codebase.Once we establish that the input is valid text, we could then ask several independent questions:Those map naturally onto Noul queries.Conceptually:Jev might return probabilities such as:Our application then decides what to do next. Perhaps:Jev makes the semantic judgement. Our program makes the operational decision. Before presenting the full example code, we’ll need some test input data. I had ChatGPT create two text files. One contains PII and confidential information, and the other doesn’t. I also sourced a .png image file and a PDF file to make sure the document classification side was working too. Here is the full code. Before running it, you’ll need an API key; set it up in a .env file. Use the .env.example file in my GitHub repo (linked at the end) as a template. You can get a key at,https://console.typesafe.ai/keys. NB: As far as I can tell, model access is free for now. I didn't need to enter any credit card details to get an API key.I first ran the classifier against the text file that contained sensitive data. Here is what that file looked like.As you can see it's full of private, sensitive data. And this is what Jev returned.That’s pretty conclusive, high probability on most measures.Here are the non-sensitive file contents and what Jev returned for itAgain, that's exactly what was to be expected given the benign nature of the file contents. To finish up, I also fed in a PDF file and an image file. Here is what was returned.SummaryUnlike the LLMs most of us are familiar with, Jev is not primarily designed to generate language or think deeply about questions. It is designed to make structured decisions — and quickly.Those decisions currently revolve around three primitives:This makes Jev particularly interesting for classification, routing, scoring, verification and other situations where software needs a decision rather than another piece of text.Whether this approach turns into a major new branch of AI architecture remains to be seen, but the underlying idea is worth thinking about.AI labs have spent the last few years making language models better at talking to humans. Jev asks what AI should look like when the thing on the other side isn’t a human at all — it’s another program.All the code and input files I used for this article can be found in my GitHub repo at: https://github.com/taupirho/jev-testThe official Jev announcement page can be found at:https://typesafe.ai/blog/introducing-system-one-models-and-jev