Jerry Liu:这是一篇我们研究团队关于校准置信度分数重要性的精彩博客文章
这是一篇我们研究团队关于校准置信度分数重要性的精彩博客文章。
在这里,它处于文档提取的背景下,它对于更一般的智能体决策也极其重要(例如与 Jev 一起)
当你设置置信度阈值时,你可以选择自动接受高于该阈值的值,并对低于阈值的值进行人工干预审查。
置信度阈值越高,你能保证的精确度就越高(例如,0.7 的置信度可能意味着 95% 的精确度,0.9 的置信度可能意味着 98% 的精确度),但当然,你需要对假阴性进行更多的人工审查。
我们投入了大量工作,以确保我们的置信度分数得到良好校准,并真实反映生产环境中复杂文档的不确定性。
来查看我们的博客:https://www.llamaindex.ai/blog/what-makes-an-extraction-confidence-score-useful
LlamaParse:
对照原文
This is a fantastic blog post from our research team on the importance of calibrated confidence scores. Here it's in the context of document extraction, it's also extremely important for more general agentic decision making (eg with Jev) When you set a confidence threshold, you can choose to automatically accept values above that threshold and do HITL review of values below the threshold. The higher the confidence threshold, the higher precision you're able to guarantee (e.g. confidence of 0.7 could mean 95% precision, confidence of 0.9 could mean 98% precision), but of course the more human review you'd have to do on false negatives. We've put in a lot of work to make sure our confidence scores are well calibrated and represents real uncertainty over complex documents in production. Come check out our blog: https://t.co/Mel0ScoCBv LlamaParse: https://t.co/XYZmx5TFz8