NUWA EXTERNAL ASSESSMENT ·
SEC-bench Pro official snapshot
Published evaluation results collected from SEC-bench Pro, retaining the reported systems and configurations.
Published Not verified
Sources ↓Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
Models covered
SEC-bench Pro · 260617 / overall · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / overall · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / v8 · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / v8 · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / v8 · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / v8 · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / firefox · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / firefox · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / firefox · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / firefox · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / linux · headline
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SEC-bench Pro · 260617 / linux · completed
Working proof-of-concept reproduction on a versioned software-security task set.
Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator.
External evaluation:
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Report coverage
| Section | Source location | Treatment |
|---|---|---|
| SEC-bench Pro · 260617 / overall · headline | /0/results/0/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / overall · completed | /0/results/0/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / v8 · headline | /1/results/0/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / v8 · completed | /1/results/0/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / v8 · headline | /1/results/1/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / v8 · completed | /1/results/1/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / firefox · headline | /2/results/0/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / firefox · completed | /2/results/0/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / firefox · headline | /2/results/1/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / firefox · completed | /2/results/1/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / linux · headline | /3/results/0/score_modes/headline/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
| SEC-bench Pro · 260617 / linux · completed | /3/results/0/score_modes/completed/score | reviewed-source-group Headline and Completed apply different inclusion rules and may change both numerator and denominator. Provider and backend labels do not identify the actual evaluator. |
References
Numerical results are reported by the cited authors.
- [1] SEC-bench Pro official leaderboard: selected 260617 and historical 260505 ; /0/results/0/score_modes/headline/score; /0/results/0/score_modes/completed/score; /0/results/1/score_modes/headline/score