def gate(items, limit):
return all(x["score"] <= limit for x in items) # 空集时循环体不执行,函数照样返回 True
def gate_guarded(items, limit):
assert items, "empty set: nothing was checked" # 空集守卫:没有对象可检时必须失败
return all(x["score"] <= limit for x in items)
print("unguarded on empty set:", gate([], 5))
try:
gate_guarded([], 5)
print("guarded on empty set: PASS (unexpected)")
except AssertionError as e:
print("guarded on empty set: FAIL:", e)
print("guarded on real data:", gate_guarded([{"score": 3}], 5))
预期观察:第一行打印 unguarded on empty set: True,说明没有守卫的判据在空集上恒真;第二行打印 guarded on empty set: FAIL: empty set: nothing was checked,说明守卫把「没有对象可检」变成可见失败;最后一行打印 guarded on real data: True,说明判据在真实数据上仍按预期工作。
filepython
查看代码
rows = [{"id": "a", "ok": True}, {"id": "b", "ok": False}]
scope = rows[:1] # 判据只覆盖第一个对象
print("gate over scope:", all(r["ok"] for r in scope))
print("claim over all rows:", all(r["ok"] for r in rows))
预期观察:第一行打印 gate over scope: True:判据只覆盖一个对象且存量数据恰好不触发,于是显示全绿;第二行打印 claim over all rows: False,说明结论所断言的范围大于判据覆盖的范围时,全绿不构成达标结论。
Deterministic eval metrics: equals / ... Model-assisted: similar / classifier / moderation / llm-rubric / g-eval / ... Every test type can be negated by prepending `not-`
Not every success criteria or failing eval is best solved by prompt engineering. For example, you can sometimes improve latency and cost more easily by selecting a different model.