Name: Webinar: From “Prompt and Pray” to Precision: A First Look at aji GenAI Review
Uploaded: 2025-10-09T17:01:05.580Z
Duration: 46 min 48 s
Description: Webinar: From “Prompt and Pray” to Precision: A First Look at aji GenAI Review

Transcript for "Webinar: From “Prompt and Pray” to Precision: A First Look at aji GenAI Review": Here you go. Alright. Hello, everyone, and welcome to this review webinar. I'm your host, Robert Hilson. And I'm joined today by the man, the myth, the legend. According to him, Ash Patel. Ash, thanks so much for, being here, man. We appreciate it. Thanks for your time. Happy to be here. Thank you. It's fine, Drew. It's always a pleasure. And welcome to, a couple 100 people that actually showed up to us. We really appreciate your time today. We are very excited to show you a product that we've been developing for, a couple years now, and as of last week is finally out in the market. You can use, Adji today. That's that is Adji. It's our GenAI review engine. And so, we're excited to show you a demo, today. The way that this program is is gonna work, I'm gonna tee up the conversation with some observations around kind of, what we and other practitioners are seeing with respect to, limitations and challenges associated with kind of those first wave of GenAI review solutions, and then we'll get into a discussion of how and this is really gonna be the gist of of, Ash's presentation, how we seek to overcome those limitations, and those challenges, with Ash. And so, I invite everyone here, and, again, there's there's a big group, so we'll try to get to everything that we can. But please ask questions. This is your forum to, ask about Ash, in in a in a way that you're not, you know, having to talk to a salesperson. So, we wanna make this conversational. And, if we get a question, I'll make I'll make sure to, to break in. So, with that, let's, let's throw up some slides briefly. And I I said I wasn't gonna share our our bio slides, Ash, because, both of them are are are are pictures on the slides are at least 10 years old. So we'll so we'll skip that one. But but to just kinda kick off the conversation, I think the pitch the pitch around GenAI review is is pretty elegant. Right? And this is a, obviously, a layman's description of it, but, you know, you have you have your your uploaded documents, you write a prompt, and you get back coded results either for responsiveness or issue tagging for for privilege. There's obviously a lot going on under the hood with respect to how the LLMs are working, how how, the prompts are are pairing with results, but that's that's the general premise. In talking to practitioners though, and in our research building Adi, like, we consistently heard four primary concerns. And, Ash, feel free to jump in here. But but those are in order. First, this this problem of kind of the AI black box. Right? When AI classifies the document as either, responsive or nonresponsive, for instance, there's sometimes limited visibility into the how and the why. And, obviously, like, if you're in a position if you're ever in a position where you have to describe, like, the the underlying logic to an opposing party, to your own client, to, attorneys at your own firm, to, you know, God forbid, a judge, like, that unknown can be a deal breaker. Second is this concept of, prompt fear. And and prompt fear, it it leads to a behavior that we've I don't know if we pointed this or is out there in the market, but it's a it's a funny phrase, prompt and pray. Right? And the way we kinda see that is twofold. First, prompting is just a a fairly new concept. And so, generally, there's some discomfort with it, especially for attorneys. And I I was actually talking to, somebody the other day, attorney at a big law firm, somebody who's, very well respected in the industry. And he's like, look, Robert. Like, one thing you gotta understand is, like, attorneys do not like to do things that they're not good at. And, right, and and prompting is, something that a lot of us aren't good at because we just don't have a lot of experience with it. The second thing, kind of related to, prompt fear is this idea that, you know, you're basically, faced with this blank screen, and the objective is to kind of come up with this perfect prompt, or or definition, that is going to result in, you know, kind of the the perfect set of documents returned. And if if, and if it doesn't come back, you know, in in the way that you expect it, like, you're going to be paying dearly for it. So in many cases, what do you end up doing? You have to rely on, an expert, you know, prompting experts. They'll need to refine these these, these prompts before you actually, you know, set them loose against the greater document set. Or, you know, what we're hearing is that, you'll actually have to rely on tools like a chat g p t or a cloud, to get familiar with with the prompt and kind of how they're going to work against, again, a a greater document site. And then you'll port those back into the review module. The third, and this is probably pretty obvious, but it's, you know, it's the idea that GenAI approaches, for now can be cost prohibitive. And a lot of that might arise from, you know, calibrating your definitions and searches before you actually run them against the greater document set. Per document pricing models have become, the standard pretty much across all tools. Adi basically prices on a per document model. And so, you're gonna get charged even if it's a minimal, a minimal cost, even for all of the testing that you're doing before, again, you actually, you know, set the definition loose against the against the wider document set. And so the the thinking is if we can deliver a way to get you to confidence faster, and with fewer docs, that is a win. And then finally, there's, there's a challenge about around basically, like, GenAI can be an all or none approach when it comes to review. Either you're using GenAI to review everything or you're using an entirely different approach. That obviously doesn't take into the account, take into account the fact that cases are built differently and not only you know, cases are built differently, but but documents within cases are different. And there are different, and and and better ways to get, you know, and review, different documents. So, those are kind of the four challenges that we sought to overcome in building Adi and kind of delivering a, you know, a better way that not only does things better and faster, but is going to deliver more accurate results with more transparency, at a lower, total cost. Ash, before I get into kind of the the four ways that we we sought to overcome those, like, do you have any do you have any thoughts on kinda what we just went through? No. I I think when I start showing everything we did with our product, I think there's a first generation pass automated pass review that, you know, a lot of competitors tools devised, and we took a lot of this feedback that you are highlighting here to incorporate into our workflows, into buttons you click, into things that, you know, would make sense where unlike a lot of other tools that are out there where you also have to become an AI expert, a data scientist overnight, a prompt engineer overnight, we are trying to simplify kind of that burden on the end user who is an attorney. They have their day job. Their job is to be either a project manager, an attorney somewhere in the litigation cycle to more accessibly and easily use these tools addressing the current concerns you just highlighted there. Yep. Well put. So, to then kinda kick off this, this this demo that we're gonna be doing, we build Adigi to address each of these challenges, and I'll just kinda briefly run through these. The first is, like, we wanna deliver more transparency into the block, into the black box. And so we built Adi to show its work. And what that means is that every classification decision, it's gonna come with detailed reasoning, and then direct citations in some instances, to the relevant portion of the doc, which means you're not just getting, you know, a yes or no answer. You're getting the the underlying why with material that, you know, that is sourced that you can use to actually validate the decision. The second piece, we're we're introducing what we call, the AI advisor. This is a a feature that is patent pending, and it it basically helps you refine your prompts or your definition, within, Adi itself. It's gonna give you recommendations for creating a higher quality profit. It's gonna return higher quality results, and you can do that without having to do testing, you know, in a different module or in, ChargeBT for instance. To deliver better cost control, one of the things that we were really kind of focused on is how can you get to calibration and and validation of results faster. Like, how can you, test the AI and be confident that it's gonna deliver, deliver the best results in, while reviewing the fewest number of documents. And then finally, to kind of solve for this, all or none problem, either, you know, you can use Gen AI review for everything or you have to use a different solution entirely. We've enabled something that we call hybrid hybrid mode. This is also patent pending, and it, it it kind of allows you to dip your toe in the Gen AI waters with, you know, technology that is, not only widely used in in, in in this profession, in this industry, but it's actually been, you know, given a a rubber stamp by the by the courts as well. So, with that as lead up, Ash, I'm gonna shop share my my screen. I'm gonna turn it over to you. And, again, one more time, I'm gonna invite, everybody that's on here. Please, like, ask your questions, and, let's let's, let's make this conversational. Absolutely. So I am gonna go ahead and start sharing my screen right here. First thing I always say, anytime I'm sharing a screen, anytime I'm doing a product demonstration, look, we appreciate your time. We're happy to have you here. Thank you for taking time out of your day to kinda join us here for this webinar. And feel free to ask questions. Right? Interrupt us. Ask questions along the way. The more interactive it is, the better off it's gonna be. One thing look. With Ash, I'm highlighting right over our review window right here. Right? With all the visualizations you get, with all the modules you get, we placed it into a module right here. And that's what we're gonna really walk through right here today. Right? How does this work? How am I leveraging some of the things Ravi discussed in terms of things we've seen as feedback in the industry? What did we do to incorporate them, tackle them within our workflows right here? But before we do that, one thing I really like highlighting here just just at the start of everything is, what is the end result? Right? After I do this, why is this significant for me as a practitioner, as a project manager, as a reviewer right here? What is my life gonna look like when I run an AG review right here? So I popped open this document right here, and what you're gonna see is when you create a workflow, Reveal is gonna come over the top and start giving you reasoning, rationale for every single kind of definition you put in that you wanted to review your document dataset for right here. So one thing you're gonna see is, you know, request 13 right here, and I'm gonna walk you through these two, says, hey. Reveal didn't find anything related to this request. It is very much built and architected to mimic those workflows attorneys really have in real life. Right? Document requests, requests for productions right here. People asking one or the other side for the documents, the obligation to produce. And you can see the rationale, the reasoning behind the yes or no right here. And, specifically, with the yes, one thing you're gonna see is not only are we gonna tell you, hey. What did Reveal find for the specific request you have right here, but where did it find it? Right? So you can then go in and validate it. Right? So these footnote citations. Now one thing that's being really released with Reveal is what we call Ask as well on a document level right here. We released our Ask feature two years ago. Pretty exciting then. It was our kind of anchored LLM in the sense of it's a GenAI chatbot but anchored to your dataset right here. So this idea that if I ever have a 24 page PowerPoint, 50 page, you know, document right here, I can come in here and say, let's summarize this document. Give me the footnotes that I really need to know from a review standpoint before I start kind of really, you know, analytically, legally slicing and dicing this document right here. So ask Ash really kind of work well within the Reveal ecosystem. I'm gonna show you how they pair up really well here. So, Ash, just to level set here, you are gonna come in and create a workflow right here, which is you are gonna go through this whole module of coming in and saying, hey. What is the name? What's the project overview? What do we want Ash to do when we're kind of coming in? We have a corpus of documents we ingested in. We have a dataset we really gotta get reviewed real fast, quick turnaround, or you have an existing workflow going on. I'm gonna go into one right here just to show you the things you would type in for an Ash review workflow. Right? The first thing is you're gonna come in and give it a name. Alright? Any context you're providing right here from a project overview or a contextualization standpoint, it's helping. It's teaching the Hilson to learn what these things are that it needs to know for the review that's about to happen. The key thing when creating an Azure workflow is coming over and giving it a definition. You can have as many of definitions as you want right here. The real idea and kind of what Robert was alluding to is you get this. Legally, you get something like this when you're doing a request for production or a document request. Right? These are things we always see is give me any and all documents related to x, y, and z with not overly broad, not overly burdensome language right here. And this is the thing you're expected to produce or give to the other party right here. You can take that exact workflow, right, as a legal practitioner and put it into the workflow for Aji. And what we've really done is come in, and what Robert alluded to is put in a definition AI advisor over the top. Now what this does is come in and look at that sentence. Right? And from a review standpoint, from actually understanding the sentence, start breaking it down and tell you, hey. What are the things we think you could improve upon? Because what we're doing here is taking this literal definition or prompt and saying, we want you to go look at this inside of the dataset that we ingested in right here. The destination AI adviser comes back with a few things right here. This one in particular, I love when it says identified here because I think this is, like, the whole kind of embodiment of a reel right here where it's gonna break down the sentence. Right? But more than that, we have kind of brain space features. We're gonna conceptually break this down. We have next LP features. Right? Entities, things that we're telling the computer, the LOM, hey. We want you to go look for these things. And really quickly, it's just telling you, hey. This is kind of what you're telling the system to go look for right here. Right? The AI advisor goes beyond that and starts saying, hey. There's a few other things we think you should consider in this sentence right here. I'm gonna go through a few of these right here today. More than happy to go through more. But the idea here is as you type in, you know, your prompt, we call them definitions here, we're gonna tell you, hey. Add in clear kind of scope and boundaries right here such as a time frame for this sales data right here. At the top, this definition is looking for any and all sales. Right? Buy for electricity related to the California Department of Water Resources. So the AI advisor is coming over the top and saying, maybe you wanna include a time frame for when that sales data is gonna come back for here. Clarity and precision is another one right here. Right? And every time we're telling you, hey. Maybe you wanna clarify all sales right here. I think one of the key notes and what really helps kind of that onboarding of doing an Azure review workflow is we're telling you why do we think this is gonna be helpful right here. Right? If we're telling you to clarify all sales right here, this is helping the system understand the level of details required for those documents. So each little kind of sentence bullet point is gonna come paired with what is this gonna be helping me for my end goal as I run our Azure review workflow right here. Last couple kind of here is structural integrity. So create a list maybe or a classification and AI nuance. Right? You say document types right here, but maybe if we start specifying the document types, we know we're the experts, we're the attorneys, we're the project managers of what the other side may be looking for, what our client may be willing to produce right here. Right? Coming in and saying, specify the type of documents right here. AI advisor is meant to help you kind of understand, hey. Not only what's good about the definition or the prompt that you're putting into the system for the Azure review workflow, but also what are the things we could improve upon and why. Right? As the system identifies documents, what are the things that we could do better right out of the gate in terms of what can we do with this prompt to come in and look it up right here? I'll pause there. Ravi, any questions or anything right here? Yeah. We we do have a bunch of questions. And going back to, and and we'll probably get to this later on. There's probably more appropriate topic, but I'll just jump in and ask it. There's a question around, confidence levels and what is kind of generally considered, industry standard, and there was a suggestion around 70 or 80%. I can I can tell you there's a difference between kind of what we hear and, like, what you see in ESI orders, for instance? And I I think around technology assist review, like, generally, the standards are around 95%. But at I mean, as you're the attorney, like, what do you see? Yeah. And I think that's always gonna be come down to that ESI protocol that you negotiate with your two respective parties. Right? And I think there's a general, like, substandard of, you know, ninety ten and ninety five five, and it depends on what metric you're talking about right here. I think the nice part is is this is the legwork. Right? Everybody does this. This little configuration button, if you think about datasets you've ingested in and a room full of contract attorneys or the associates that you have to train up, this kind of workflow right here always reminds me of my project manager days when I have to create an instruction protocol right here. Right? I have to teach 30 other people, hey. What's important in this case that we're looking for? And in this instance, what we're really doing is taking that same workflow you're used to. Right? Whether it's a roomful of associates, partners, attorneys, roomful of contract attorneys, and saying, what do you want kind of Aji to look for in this dataset? We're teaching Aji right here. We're gonna get into the actual metrics when we start talking about calibration of what you can define what you want for the stop and go right here. I'll quickly wrap this up with the definitions is key part. AI advisers is the key part right here. We give you kind of the breakdown of what's good, what could be better. Right? It's not that prompt and pray, go around it on 10,000 docs, see what you get kind of right here. It is that idea of, hey. Let's tweak this a little bit because we really want you to define what the system's looking for right here. These next few things, you know, in terms of adding context, adding people, organizations. Right? The more context you give, Reveal, the better it's gonna understand. What is this dataset? Are there code words related to something here that I should be on the lookout for? Right? You do all this. This is kind of the starting point. Right? When the next step comes in is you come in and do a calibration right here. So the idea here is you test those definitions that you just ran with Reveal. And And what that means is you come in over the top. You can see I ran a couple right here. Right? I'm just gonna click this add button. And what happens here is there's a couple things you can decide to do. Now in terms of the workflow name, to me, to everybody kind of in the industry, this calibration, the way we've integrated it in, is testing your definition against what are the documents and who's gonna be looking for it. Our GenAI review, our Reveal is gonna come over the top and start coding the documents, and then a subject matter expert, a human's gonna come in and be like, this is good. This is bad. You know? And Reveal learns the most, and I'm a show example of this, from when they're strong disagreements or when Reveal actually thinks, hey. This is a borderline call. We have a flag for that. Right? Where it's like, hey. Maybe we could get a strong yes or a strong no, and it start gonna give you definition updates on that. This workflow right here. Right? We wanted to mirror people have their industry's true protocols for technology assisted review, continuous active learning. Right? They have their TAR protocols. So you can come in and use an existing document selection. You can create a new document selection in this entire pane right here. Right? So the module's meant to be, hey. You can do it all in one screen right here with different tabs. Right? This button right here, suggest documents automatically, goes back to what I showed at the beginning. We had a tool we released two years ago, Ask. It's gonna take your definition. It's gonna run it against the dataset, gonna give you a good mix of things that hit on it, low probability documents. Right? Kind of that control set that we're used to right here. But it's leveraging ASK to come in over your dataset and say, you know, these are the documents we floated up and bubbled up to the top. The really nice button here is this little radio button at the bottom. When you run a calibration, you're teaching Ash, hey. It's looking for stuff. It's gonna code things automatically right here, but you're actually teaching it, this is good. This is bad. This is a strong negative right here. And when you click auto tune definition based on results, it's updating that definition in real time or giving you suggested edits based off what that human coded versus what the Gen AI coded right here. So in real life, what this looks like is look. I ran three right here. Here are my agreement rates with the Aji coding. Right? I'm gonna go into this one that I coded 50 documents for right here. And what I mean by everything I showed you is you're gonna get these tried and true visualizations we try to give you in Reveal. Right? Here's your kind of donut in terms of what did GenAI code these documents. What did the subject matter experts or the human code these documents? What are the metrics surrounding it, right, whether false positives, false negatives? What's my agreement rate? And the idea here is you test your definition until you get to an agreement rate that you like right here. You can come here on one screen and actually see all of your documents. You can filter on them. I could pop this out into another window if I wanted to. Right? And the whole point is you do a definition. GenAI is gonna come in. Ajay is gonna come in and give you a rating right here. Human reviewer. Right? Subject matter expert. Maybe the attorney that's got the case or the lead attorney comes in and says, here's what I'm really looking for. Right? And the stronger the agreement or disagreement between Aji and the human, the better the starting point for Aji. That's the key distinction. Right? We're testing our definition. The yes yeses are great, but a strong no and a strong yes, that's teaching Aji, okay. This is what we're looking for. A colleague of mine always used this example just to bring it back to If I'm teaching a computer, if I'm teaching the LOM to go look for oil in the dataset, right, it's gonna come back and give me face oil, cooking oil, gasoline, any type of oil right here. When we come in and start specifying, hey. I'm actually looking for gasoline right here. It's gonna understand that context. The manual decision, the human coating first with the Gen AI coating, finding that kind of positive, tweaking them, and actually getting this little auto tune definition right here is the key distinction in the calibration right here in teaching Aji, hey. What are we actually looking for? And this is what kind of Aji came back after my three runs that I did in the calibration. It's telling me, hey. I know you had one sentence right here, but you're actually looking for all of these things. It told us to include c w CDWR right here. Include the acronym. Right? You are actually looking for hourly porting of sales based off a strong yes, no right here. Right? And at any point when you're doing this, you actually have control of, I wanna see all these documents right here. I wanna come in and say, I wanna see these documents in the dataset. I wanna filter on the Gen AI review rating. I wanna figure out the manual review rating. Right? All of this within one module on the back end to pop out and start actually reviewing the documents or start saying, hey. Show me all those strong yeses or no. Show me those borderline calls so I can clear those up and get a stronger kind of calibration when I'm ready to run a JNI review rating right here. I'll pause here again. Kind of a few questions. Yeah. We we have some from the audience and I have a couple of things to drill down on as well. Ash, maybe you can help kinda give some more context for. So, you mentioned there's this this tie in with, with Ash, kind of our our fact finding Gen AI tool. And and just to kinda clarify what you what you said, because I this took me a while to understand. What ask is identifying within the the larger documents that are not just documents where it thinks there is strong agreement with with the definition, but it is taking a a sampling of of documents where it can say, these are, you know I believe this is in, strong agreement with the definition. This is not an agreement. This might be borderline. And then you're you're basically lining those up against the human coding decisions, and that's how you're getting to that's that's how you're getting the confidence. Is that right? And I I guess the the yeah. Go ahead. 100%. Yeah. No. A 100% is when you're using that suggest documents automatically, you're getting that kind of what we call in the industry. Right? Diverse active. Right? Everything that kinda hits on the definition. Right? But then that that's where the algorithms come in, and we're giving you that active pool of low level documents that might not hit. But in a true control set, you're always wanting kind of a different what's the true universe of my document right here? Not just everything that may be a yes. Right? So 100% in terms of how you articulate to that. Okay. Got it. And then and then in terms of, what what documents ask is actually pulling, what's the what's kind of the underlying like, how does it know to pull certain documents versus others? Yeah. So with Ask, it's gonna break down your you know, so that goes into the whole kind of the LLM, vectoring, embeddings. Right? That actually is the underlying techno technology with the LLMs. Right? Is Ash is leveraging that. We built out an entire semantic index the minute we ingest in your data, right, among many things, right, with our concept searching or our cluster wheel or anything like that. That semantic index we're building when we're ingesting in those documents, that's really what ask is running off of right here. And, again, that brings in underlying technology with the LOMs of what are we looking for for the context of the words, how do we vectorize them, how do words relate to each other in a numerical representation right here. But a lot like, you know, concept searching in here. It's looking for the words in a definition, in the dataset, and other words or documents related to that. Got it. That makes sense. And then, in terms of, how the calibration works and, like, the the number of rounds that you might need to Yeah. You know, to do to get to confidence. One question would be just, like, what is the what would be the alternative to doing this? And, like, one thing that that we've heard that I've heard, for instance, is in the context here is that a lot of, you know, a a a lot of competitive tools, for instance, the module exists kind of separate and apart from, you know, the rest of the review platform. So what you might have to do, for example, is is, you know, write a script for a random sampling of documents and then kind of bump that against, you know, your your your your results for, you know, in in Geni because there's no kind of asset equivalent. But can can you kind of explain that more? Like, if if you're not doing calibration like this, what would it otherwise look like? I mean, if you're not doing a calibration like this, right, it is going back to what we started with. But this you are really hyper focused in on this definition, making sure it's a 100% accurate, and then you're tossing it over the wall saying, alright. Go review all the documents like this. You never really get a test. It's never that feedback loop of, am I on the right track even? Right? And then in real review, what do we do is in the first week of review pre prior to GenAI coming into play right here is that first week is that learning module right here. Right? Reviewers may find something responsive, nonresponsive. You give them issue tags right here, and they're like, I'm not seeing anything related to the responsive metrics that you put out here. That's that first week right here. That's what we want calibration to be really, is you come in, you have your instructions, you have your definitions, test it on the back end before you actually run it on an entire dataset or your corpus of documents. So you get that feedback and also have your subject matter experts or your human reviewers come in and be like, yep. This is right. This is wrong. And Ash learns the most from that with the auto tune kind of definition updates right here. Got it. That that makes sense. And then there was a specific question around, like, how many rounds of calibration does it typically get so that, you know, you're confident that the this is gonna return the right results when you run it against the broader set of documents. Yeah. And I think that will always depend. It's my favorite phrase to always say, data set and size. Right? But, like, the truth is, look, if you have a corpus of documents that's 10,000 documents, 20,000, 50,000 documents, The idea here is you're trying to get to maybe an above 90% agreement. Right? We run it over a 100,000 documents. We're trying to get to a better agreement rate or something that's acceptable right here. Again, this part right here is really you calibrating. I I teaching is the word that I always use here is this is when the human comes in and teaches Ash. Right? We're actually teaching the LLM. This is what I'm looking for in the dataset. So the agreement right here is kind of that idea that am I getting on the right track? Is I actually actually understanding and picking up what I'm looking for in this dataset, George, to tweak that definition a little bit to get to the right spot. And, again, as I'm coding, if I checked off that little auto tune button, I'm not the one that has to come up with the definition. Our module here, our workflow is gonna help you with the agreements and disagreements come up with that kind of tweak definition. Got it. Okay. There's another question around and, Jason, we might need a little bit more context for just just to make sure we're answering the right question. But, where is, the document content actually actually coming from? And actually the I think the question oh, no. It's here. Where is the document contact coming from? Public Sphere, our DMS, or both? I I mean, I think the answer is it's coming from the documents that that are in your review side, but you Yeah. A 100%. Yeah. I'm I'm assuming DMS is document management system, but it is whatever you ingest into a project. Ashi's anchored to the documents you ingest into a review project right here. So I'm in this one right here. Right? It has about a million documents. Your project could have 50,000 documents. That is this kind of universe, the corpus of documents, Adi can run against. And then even within that, maybe you're not reviewing everything in that thing that you ingested in. Maybe it's the search terms you and the other party agreed to. Right? It's ultimately what you point and tell as you, hey. I want you to go review or want you to calibrate against this. Awesome. Couple more. And by the way, thanks for the great questions. Keep them coming. We'll we'll answer as many as we can. And then, Ash, we'll we'll, we'll turn back to you and and get through the rest of this. Okay. Nathan, great question. Does Adi limit the number of definitions that can be run at one time? I believe the answer is yes, but it but it's it's a big number, Ash. Yeah. So right now so as we release it right here, it's gonna be, I believe, 10. Then it's gonna exponentially increase really quickly. We are kind of releasing it right now, and the idea is run as many as you want, and that's where we wanna get to. Yeah. Awesome. And there was one other question around, basically, how do you approach, you know, a review where you know what you're looking for versus an investigation. I'm I'm losing that in all the other questions. So, Ash, we'll get back to that one, and I'll turn it over to you. Yeah. No. In that sense, it is a review. You know what you're looking for. You probably really have that definition down line. Right? A review where it's an internal investigation where you're investigating. I think we have a lot of tools just like how I you know, ask is gonna come over the top and suggest documents automatically. It's like, hey. We have an idea of what we're looking for. Can you float us? Can you prioritize some of the documents that may be hit? That is a 100% of time that I'm hitting. Suggest documents automatically because I only have an idea about what I'm looking for in this dataset here. So with this kind of and then you got any more questions here before I go into how we can deploy this out? Yeah. We have several, but but why don't why don't we why don't we get through the rest of the demo? Yeah. 100%. No. That makes total sense. So, look, calibration is kind of built into this workflow right here. Right? Taking the feedback from our customers, from our clients being like, we don't wanna hop into a million other screens. Logical next step here is and this is truly where you get the flexibility. Right? When you come in or when you wanna do a GenAI review, you have the ability a two parter here, and I think this is where our data science team really kinda knocked this one out of the park is it's up to you ultimately. Do you wanna run a full blown GenAI review, meaning, Adi comes in? And that's why I start with that end document. Right? It takes your definitions. It gives you rationales. It tells you yes, no for every single document in a review. How you would do that is click add right here, and you could come in, name whatever workflow that you want to, right, and say, hey. I wanna exist, create an existing document selection, or create a new one from the back end right here. Right? That idea that I don't have to hop through portals. I can pick a safe search. I can pick wherever my document universe is. I calibrate it. I'm ready to go. Here's the thousand documents. Here's the 10,000 documents I want as you go review right here. Right? That is gonna get you, effectively, when you come in here, something that looks like a lot like this, which is this donut that comes up as you came in. It reviewed everything. Right? You're gonna be able to see your yeses, your noes, your donuts. If I clicked into one of these, it pop them into my review window right here. But that idea that I can come in and say, hey. What are the documents that got hit on right here, and what did it find as a borderline? And then at any point, you can use all our filters back here. I could preview the document. I could pop it out open right here. The whole point is we want you to have a seamless experience and not go from module to module. Right? I can filter on all the reviews right here, review coding that Ash did from this GenAI review rating. The flip of this is you can come in and leverage what we call our hybrid GenAI review right here. Right? And what this is is if you come in, we have our kind of industry standard tried and true. You know, I always say jurisdictionally accepted. And somebody asked this question earlier. Right? What are the metrics that I'd be using? Well, what's the metrics you agree to or what's kind of what your company or law firm likes doing is what are the definitions in terms of precision, recall, f one score that I wanna leverage right here? You would be getting a report just like this. The idea here is you can come in. When you kick off an ASGI workflow, we are kicking off a classifier. You can come in and use that classifier for a small batch of documents. Right? ASGI is gonna review those documents, give you those reasonings, and it's gonna start building up that classifier right here. When you're ready to, you can come in and say, okay. I wanna use that classifier, and I have a stopping condition right here. I wanted to go in and check batches in and out of, you know, 10 rounds of 40 documents or 10 rounds of 400 documents. This is ultimately up to you. This is the flexibility we're trying to provide to the end client. If I click this little button right here, this is kinda really getting into the meat of those metrics we were talking about right here. If I have a probability score that I want Ash to stop reviewing at, once the classifier goes in, it's gonna start reviewing every single document. I can pipe that in right here. If I have metrics that I really wanna target right here, batches that I wanted to do until it hits a minimum recall rate, until it hits a precision rate that I want, or both. Right? Ultimately, it is up to the end user for whatever workflow they have, whatever case walked into the door, and whatever they negotiated with the other side that, hey. We're gonna review all the documents until we hit x, y, and z metric right here. Right? So being able to come in, not only, you know, say, hey. I want Ash to go do a full GenAI review. Right? You have that ability to do that. Come over the top and maybe you wanna come in and say, hey. I actually wanna start using that reviewed probability score, and I want Ash to review until it gets to 70%, and then I'm gonna get something else involved. I'm gonna get humans involved. You have that ability to come in and leverage our hybrid mode or kind of our full GenAI review right here, Both in the background right here. Both ready to deploy whenever you're building out a workflow within a Aji review workflow right here. Pause there again. Questions? We have a bunch of questions. And as I mean, one of them was generally around, like, what is the best approach for potentially calling documents before you actually, bring in Adji? Yeah. Well, or another way to ask it is is that the best approach, or should you just start off with Aji and kinda let it go against everything? That's such a good question. Ultimately, it's always gonna depend on the corpus of documents. I think in the Reveal ecosystem, we really have a lot of great tools built out for the end review right here. I keep alluding to ask. I'm just gonna flip quickly back from this review screen into the review. What we've seen clients do is come in. They have an idea of what they're looking for in the document in the dataset, but then they're coming in and saying, hey. You know, our data scientist team architected every one of these tools. We want you to not to feel like you need to be a prompt engineer or data scientist. But if you have some of the know hows, you can come in over the time and say, hey. Have this question. This is what I really want in this data set that's coming in. After the answer, provide me 10 Boolean searches or search terms, and they use that as kind of how they're gonna kind of start doing targeted searches. They're calling kind of their document corpus right there, and that's the subset that they're either reviewing human wise or kind of piping straight into Azure saying, I know this is the stuff I wanna look for. This is an internal investigation. That's how I'm gonna call down on it. Or even in a broader scope. Right? Leveraging something like ask to come in, get your search terms if you don't already have them, and then going into the subset there and leveraging ASHI. Right? Or out of the gate, you could come in, build that classifier, and you ultimately control kind of, am I doing a full Gen AI review? Am I coming in and actually doing a hybrid review to start figuring out the classifiers and the probabilities that I want right here? Other questions? There's a question, generally around which of these features are coming to Logikcull, and then is there any forthcoming integration between, Reveal and Reveal? And the the context is basically, you know, I can imagine a scenario where I wanna do a bunch of calling and then run Adi. I I think I didn't speak to that one. Yes. Adi both Ask and Adi, by the way, are are coming to Reveal. Those are probably, I would say, maybe next year, types of things. And then we are also building a connector. Again, this is a 2026 thing directly between, Reveal and and Reveal enterprise. It's something that we get asked about all the time, and it's, you know, it's top of mind for our our product people. Ash, let me turn it back over to you if we have anything else to conclude on, and, and I'll go through these, like, 25 questions that we No. No. I think I think I showed, I mean, the entire module, the workflow right here. And, honestly, that is like look. I know in a lot of other reviews, you're probably kinda coming in here and getting what's the end picture? What does this look like? That's why I start there. To be honest, is I want you to have the frame of mind of if you run an entire ASI workflow, what are you gonna get back? Right? You're either gonna get back that hybrid review where you can kind of categorize and wanna see all the documents that hit on over 70 as a score right here. That's gonna be a search you do. Right? If you run a full GenAI review, you're gonna get the yes, no for every kind of definition that you put in there. Yeses, you're gonna get the citations right here. I think if I had to say it, there's a lot of stuff that we did in terms of architecting what our team likes to call a second generation to first pass automated GenAI review right here. Right? I think this takes a lot of the feedback from the marketplace, builds in the calibration review, brings in the AI advisor, and then truly, I think the home run will be you ultimately get to decide from a flexible standpoint of, am I doing a full gen AI review, or do I kinda wanna use some of those metrics, those tie and true things from an f one recall precision rate with the hybrid review right here, leveraging the same exact workflow for both. Yeah. Awesome. Ask a couple more questions for you and then, and then we're gonna wrap up. Yeah. One question is, in a in a scenario where we are using Ash on only a partial dataset versus the entire database, Are we able to use the ask feature at all? Yeah. Absolutely. Okay. And there there's a comment about, there there seems to be, there seems to be no way to limit its scope to relevant documents when generating calibration badges. I would have to know more context there, but, out of the gate there is a workflow to that, but I would need to know exactly what the details about that question are. Yeah. That is it. Yeah. Nuance. Yeah. And and, Max, ping us in the in the comments, and we'll, we can follow-up with you directly as well. And then at Ash, this is, you know, a question and conversations that kinda come up. And, again, as an attorney, I I I think, I'd I'd like to get your perspective on this. So with technology assist review, we had kind of a a line in the sand, you know, court opinion with the silver more where it was like, okay. Here's kind of the rubber stamp. This is now fair game to use. And in in that in that case, federal reports and obviously that, you know, that that, kinda made its way through other other ports as well. Yep. Do do you feel like I guess the question is in lieu of having, an an opinion or an order like that, like, what advice would you give to practitioners who, you know, see see, you know, that that that this is a very powerful tool but might be reticent around, you know, hey. We don't we don't have, like, courts sign off yet. I think that is a 100% why our data scientists build out a hybrid review right here. Right? You need those court sign offs where it says this is where you come in and you can control the metrics that have been approved. Nationwide, jurisdictionally, in terms of precision and recall methods that have been used and approved throughout the system already, I definitely get, you know, the concerns around this is brand new and all that. I I'm in the boat that this is coming. This is gonna become in a year, two years, email threading. This is gonna be analytics. This is table stakes right here. It's more about incorporating it into your workflows, getting a hang of it now, and understanding how can I leverage a tool like this to more kind of expeditedly review these documents and get the workflow I want? Kind of convey that to the clients that you're leveraging right here that this is the next wave of kind of review right here. Yeah. Well put. And actually, for those of you interested interested, there are a couple of comments related to this. Go check out our ediscovery and federal courts, white paper that we did recently. And while that doesn't, highlight any judicial opinions around this topic because it there don't appear appear to be any yet. What it does show is that, there are, you know, there are, many parties in federal courts that are clearly using GenAI because it is reflected in the ESA ESI orders that are actually being, you know, entered into these courts. So, it it is being used whether there's been kind of a, you know, judicial quote unquote sign off or not. So, with that, I I think we're gonna wrap. We're we're running out of time here. I just wanna thank you, everybody again for just a lot of great questions, a lot of great conversation. As you delivered per usual, I appreciate you being here. And I I wanna leave everybody with this. If you're fleet of product specialists who would be, more than happy to show you the technology one on one and answer these questions in in more depth. And secondly, if you are a Orville enterprise customer, we are offering Adi at at no cost on unlimited basis between now and the end of the year, which means the time is running out. But, we're basically allocating you a a huge, you know, a a a a amount of Adi units for you to actually use this and put Adi through its paces to to basically prove out that, you know, it you can use it to, to effect. One of the reasons we're doing that is because we know that, we're we're not gonna get this right by ourselves. Like, we want practitioners and our and our customers, like, putting Adi through its cases. We're confident that it's gonna perform the way that we we say it is we we say it it will, and it does. So, please take us up on that offer. If you wanna learn more, go to revealdata.combackslashaji. And, of course, you can reach out to me. You can reach out to Ash. But I think we're going to wrap it there. Ash, any final thoughts from you? No. I appreciate everybody's time. Thank you. I have a slew of great questions here today, and I appreciate everybody's time participating in the webinar. Right on. Thank you, everyone. Talk to you next time. Bye.