NSFW Image to Text: The Best Scoring Detection Model

Abstract. Generative-image platforms must moderate the images their own models produce, a distribution on which conventional safety classifiers are known to lose accuracy. This report evaluates the each::labs NSFW Checker, a production image-safety API with three fixed policy operating points (moderate, strict, super_strict), on the official test split of UnsafeBench (Qu et al., ACM CCS 2025): 2,037 images across 11 unsafe-content categories, half real-world and half AI-generated, for 6,111 classifications with zero errors. On the Sexual category, strict reaches F1 0.896 on real-world and 0.879 on AI-generated images, the highest point estimates among all classifiers reported in the UnsafeBench paper. In strict no real-versus-AI degradation is observed (F1 difference -0.002, 95% CI [-0.070, +0.065]), in contrast to the degradation the benchmark documents for conventional classifiers. Results for all 11 categories are reported, including those outside the product's policy scope, together with bootstrap confidence intervals and paired significance tests between modes.

Outlier statistics

The table below isolates the measurements that depart from the distribution of results reported on this benchmark, in both directions. Each is developed in the section indicated.

Statistic	Value	Why it is an outlier	§
Sexual F1, real-world (strict)	0.896 [0.839, 0.943]	Highest point estimate reported on the benchmark for this category and source	4
Sexual F1, AI-generated (strict)	0.879 [0.819, 0.929]	Highest reported on the generated source, where every dedicated NSFW baseline falls below 0.70	4
Real minus AI-generated F1 (strict, all 11)	-0.002 [-0.070, +0.065]	The benchmark's central finding is a positive gap; the comparable SD Safety Filter gap on Sexual is 0.106	5
Recall on core categories (super_strict)	0.960	Near-ceiling recall, obtained at a false-positive rate of 0.411	6
Sexual FPR (super_strict)	0.79	Outlier in the adverse direction: the highest over-flagging rate of any cell in the per-category results	7
Spam and Political F1 (strict)	0.18 / 0.23	Outliers in the adverse direction, in categories outside the product's declared policy scope	7
Category attribution on Sexual true positives	98.5%	135 of 137 flagged images receive a sexual-family label, against a 51.6% exact match rate overall	8

1. Introduction

Image moderation was historically a problem of screening user uploads. For platforms built on generative models it is also a problem of screening outputs, and UnsafeBench's central finding is that conventional safety classifiers degrade precisely there, losing accuracy on AI-generated images relative to real-world ones. The Stable Diffusion safety filter falls from 0.833 to 0.727 F1 on sexual content when the images originate from a generator rather than the web; NSFW_Detector falls from 0.783 to 0.678, and NudeNet from 0.650 to 0.596.

A moderation layer for generative products must therefore be evaluated on both distributions, at the operating points it ships with, and at production latency. This report evaluates the each::labs NSFW Checker on the official UnsafeBench test split and reports every category, including those where performance is poor.

2. The classifier and its policy modes

The NSFW Checker is an image-classification endpoint on the each::labs platform. It accepts up to ten image URLs per request and returns a binary flag per image together with a request-level category label, so that a moderation service can both block and route from a single call. The same endpoint accepts text prompts, audio and video, which permits screening a prompt before generation as well as the resulting media afterwards. The system is proprietary; training data and architecture are not disclosed here.

The three policy modes are fixed operating points rather than user-tunable thresholds. Their intended surfaces are stated before any results are presented.

Mode	Intended surface	Policy
moderate	17+ and adult-permissive surfaces	Permissive by design. Allows artistic nudity and suggestive content; flags only clearly explicit or harmful material.
strict	General-audience products (default)	Balanced. Flags explicit sexual content, graphic violence, shocking imagery, self-harm and hate imagery.
super_strict	Child-safe and brand-safe placements	Maximum coverage. Intentionally over-blocks borderline content, including swimwear or the presence of a weapon.

Because each moderation system encodes its own policy, comparisons against a foreign label set are approximate. UnsafeBench's annotators applied UnsafeBench's definition of unsafe, which is broader than NSFW and matches none of the three modes above. This qualification applies to every figure in this report.

3. Evaluation setup

3.1 Benchmark. UnsafeBench contains 10,146 images labelled Safe or Unsafe by trained annotators (Fleiss' kappa 0.697) across 11 categories, sourced half from LAION-5B (real-world) and half from Lexica (AI-generated). The official 20% test split is used: 2,037 images, of which 777 are Unsafe and 1,260 Safe, 1,015 real-world and 1,022 AI-generated.

3.2 Protocol. Images-only input, no text context, batches of ten, every image under every mode: 3 x 2,037 = 6,111 classifications, none skipped and none erroring.

3.3 Metrics. The positive class is Unsafe. F1 is the primary metric, matching the benchmark's own choice, alongside precision, recall and the false-positive rate on Safe images. The API returns fixed decisions rather than scores, so threshold-free metrics do not apply; the three modes are the operating points. Uncertainty is quantified by 95% percentile bootstrap over images (B = 5,000, seed 42). Mode comparisons use exact two-sided McNemar tests paired on identical images. Confidence intervals are uncommon in the safety-classifier literature and significance tests are, to the authors' knowledge, absent from it.

3.4 Repeatability. A repeat run of 200 images under identical conditions reproduced 193 of 200 flags (96.5%, Wilson 95% CI [0.930, 0.983]), with flips balanced in both directions. The classifier is therefore stochastic at roughly 3.5% of images, presumably borderline content. Aggregate metrics are stable across realizations, but per-image results are single realizations.

3.5 Scope alignment. Following the benchmark's own coverage-alignment device, the product's policy scope is declared before results are shown. Sexual, Violence, Shocking, Self-harm and Hate are in scope (n = 1,033). The remaining six categories, Harassment, Illegal activity, Deception, Political, Health and Spam, are misinformation and regulatory classes that no NSFW product claims. All 11 are reported individually in Section 7.

4. Results: the Sexual category

The Sexual category is the classic NSFW task and the one every classifier compared here claims to cover. In strict, the default mode, the checker reaches F1 0.896 on real-world images and 0.879 on AI-generated images.

Classifier	Real-world F1	AI-generated F1
NSFW Checker, strict	0.896 [0.839, 0.943]	0.879 [0.819, 0.929]
NSFW Checker, super_strict	0.895 [0.843, 0.939]	0.816 [0.747, 0.874]
GPT-4V	0.866	0.827
SD Safety Filter	0.833	0.727
NSFW_Detector	0.783	0.678
MultiHeaded	0.692	0.815
NudeNet	0.650	0.596

Baseline rows are transcribed from Qu et al. Table 3, computed on the full 10,146-image dataset; the NSFW Checker rows use the official 2,037-image test split, so the comparison is indicative rather than exact. Sexual test-split n = 211 (150 Unsafe, 61 Safe).

The strength of the claim is bounded as follows. The dedicated NSFW classifiers, NudeNet and NSFW_Detector, fall clearly below the reported confidence intervals on both sources. SD Safety Filter and MultiHeaded fall marginally outside them. GPT-4V's reported 0.866 and 0.827 fall inside them, so the supported statement is that the checker matches a frontier vision model's reported level at production latency while holding the higher point estimate, not that it outperforms it.

Because the Sexual test split is 71% unsafe, a high F1 can coexist with substantial over-flagging of the safe minority, and it does. The strict false-positive rate on safe Sexual images is 0.38 overall, 0.26 on real-world and 0.45 on AI-generated. In moderate the same rate falls to 0.13 at a recall of 0.39. The safe-side sample is 61 images, so these rates carry wide uncertainty.

5. Results: real-world versus AI-generated images

UnsafeBench's central finding is that conventional classifiers lose accuracy on AI-generated images. For the NSFW Checker this depends on the mode.

Subset and mode	F1 real	F1 AI-gen	Difference	95% CI of difference
All 11, strict	0.573	0.575	-0.002	[-0.070, +0.065]
All 11, moderate	0.448	0.432	+0.017	[-0.057, +0.090]
All 11, super_strict	0.701	0.687	+0.014	[-0.033, +0.060]
Core 5, strict	0.716	0.739	-0.023	[-0.099, +0.050]
Core 5, moderate	0.508	0.574	-0.066	[-0.163, +0.029]
Core 5, super_strict	0.768	0.713	+0.055	[-0.004, +0.116]
Sexual, strict	0.896	0.879	+0.017	[-0.056, +0.091]

In strict no degradation on AI-generated images is observed. The difference is near zero on every subset, though the intervals are wide, at roughly plus or minus 0.07 to 0.10. The supported statement is therefore that no degradation was observed rather than that no gap exists. Intervals of this width can conceal an effect of a few points; they cannot conceal a degradation of the magnitude the Stable Diffusion filter exhibits, which is 0.106 on this category.

The result is not uniform across modes. super_strict shows a +0.055 real-over-AI gap on core categories whose interval reaches +0.116, comparable in size to that degradation, and moderate trends in the opposite direction. The robustness claim is scoped to the default mode.

6. Results: the three policy operating points

On the five in-scope categories the modes behave as follows.

Mode	Accuracy	Precision	Recall	F1	95% CI	FPR
moderate	0.731	0.791	0.416	0.546	[0.499, 0.597]	0.070
strict	0.803	0.784	0.681	0.729	[0.694, 0.766]	0.119
super_strict	0.733	0.597	0.960	0.736	[0.704, 0.765]	0.411

Flag counts across the full split order in the same direction: 297, 460 and 1,088 images. The dial is near-monotonic, with 99.0% of images respecting the ordering; 19 images (0.93%) were flagged by moderate but not strict, and one image (0.05%) by strict but not super_strict. Given the measured flip rate of Section 3.4, individual violations fall within the range repeat noise can produce.

Paired exact McNemar tests establish that the modes are genuinely distinct operating points rather than noise around a single threshold.

Pair	Overall p	Core p
strict vs moderate	1.3e-7	1.0e-9
strict vs super_strict	0.073	2.7e-5
moderate vs super_strict	0.32	0.96

strict is significantly more accurate than moderate and than super_strict on core categories, the latter being a deliberate trade in which super_strict purchases 0.960 recall with accuracy-costing false positives. moderate and super_strict do not differ in accuracy despite opposite error profiles, misses against over-flags. The discordance counts are far larger than the measured flip rate can explain, so these differences are not artifacts of stochasticity.

7. Results: per-category performance

Category	Unsafe / Safe	strict	moderate	super_strict	GPT-4V
Sexual	150 / 61	0.89	0.54	0.86	0.847
Shocking	114 / 117	0.76	0.63	0.78	0.839
Violence	79 / 114	0.56	0.53	0.71	0.738
Self-harm	29 / 170	0.42	0.41	0.48	0.590
Hate	29 / 170	0.40	0.44	0.53	0.423
Harassment	40 / 147	0.65	0.58	0.54	0.681
Illegal activity	88 / 84	0.25	0.27	0.59	0.780
Health	55 / 100	0.40	0.28	0.70	0.492
Deception	50 / 113	0.38	0.36	0.59	0.673
Political	91 / 76	0.23	0.18	0.74	0.780
Spam	52 / 108	0.18	0.20	0.68	0.537

Bold marks the best of the three modes. The GPT-4V column is the "Overall" value from Qu et al. Table 3, computed on the full dataset, and is given for reference only.

In strict, performance on the in-scope block is carried by Sexual and Shocking and falls on Violence, Self-harm and Hate, categories in which unsafe content is frequently contextual rather than visually explicit. Switching to super_strict raises Violence to 0.71 and Hate to 0.53. The six out-of-scope categories score poorly, with Political at 0.23 and Spam at 0.18; they are reported because selective reporting of favourable categories would invalidate the comparison.

The recall and false-positive decomposition for the in-scope categories is given below.

Category	strict	moderate	super_strict
Sexual	R .92 · FPR .38 · F1 .89	R .39 · FPR .13 · F1 .54	R .99 · FPR .79 · F1 .86
Shocking	R .69 · FPR .14 · F1 .76	R .49 · FPR .06 · F1 .63	R .95 · FPR .48 · F1 .78
Violence	R .43 · FPR .08 · F1 .56	R .41 · FPR .08 · F1 .53	R .99 · FPR .54 · F1 .71
Self-harm	R .38 · FPR .07 · F1 .42	R .31 · FPR .04 · F1 .41	R .90 · FPR .32 · F1 .48
Hate	R .38 · FPR .09 · F1 .40	R .41 · FPR .08 · F1 .44	R .86 · FPR .24 · F1 .53

The Sexual row contains the two extreme values of the table. Recall of 0.99 in super_strict is obtained together with a false-positive rate of 0.79, the highest over-flagging rate observed in any cell.

8. Category attribution

The API returns one category label per request, so per-image attribution requires single-image requests. The full test split was re-run in strict at batch size one and the returned category scored against the UnsafeBench category of each image. The observed vocabulary is none, nudity, sexual, sexual_minors, violence, hate and other.

Where the taxonomies align, attribution is strong: 135 of 137 flagged unsafe Sexual images (98.5%) receive a sexual-family category, 12 of 12 Hate images receive hate, and 28 of 35 Violence images (80.0%) receive violence. Across all flagged true positives the exact mapped match rate is 51.6%, and the cross-tabulation indicates that the residual is dominated by taxonomy boundaries rather than content confusion.

Model category	Leading UnsafeBench categories of the images it fired on
nudity (n=190)	Sexual 125, Shocking 31, Health 8, Self-harm 8, Illegal activity 7
violence (n=143)	Shocking 59, Violence 34, Harassment 11, Self-harm 11, Deception 10
sexual (n=57)	Sexual 33, Harassment 18, Violence 1, Illegal activity 1
hate (n=39)	Hate 27, Deception 5, Shocking 3, Violence 2
other (n=40)	Spam 8, Self-harm 5, Shocking 5, Illegal activity 5, Health 4
sexual_minors (n=8)	Violence 4, Harassment 2, Self-harm 1, Illegal activity 1

The violence label fires on 59 images that UnsafeBench files under Shocking, a category composed largely of gore, and sexual fires on 18 Harassment images, which include sexualized-harassment content. The category head describes the content it observes; the benchmark partitions that content along different lines.

9. Discussion

Three properties of these results bear on deployment. First, the robustness result of Section 5 is specific to the default mode, and a system selecting super_strict for child-safe placements should not assume it transfers. Second, the operating points are separated by large, significant margins in accuracy and error profile, which supports exposing policy as a mode parameter rather than as a single threshold applied across every surface a product ships. Third, the false-positive rates reported here are measured against UnsafeBench's policy, which is broader than NSFW and matches none of the three modes; they are suitable for comparing classifiers on a fixed yardstick and are not a forecast of the rates a given deployment will observe against its own rules.

Server-reported inference time was 3.24 seconds for a representative ten-image batch. Across 2,037 single-image requests it averaged 1.75 seconds with a median of 1.12 and a 95th percentile of 7.5. Batching therefore determines whether moderation runs inline with generation or as an offline pass.

Two limits apply to every number above. UnsafeBench's inter-annotator agreement is kappa 0.697, so no classifier can attain F1 1.0 against these labels, and the residual label disagreement affects all systems compared here. The benchmark's taxonomy is English-language and predominantly Western in framing, and results may not transfer to other moderation standards.

10. Conclusion

Evaluated zero-shot against a public benchmark whose labels encode an external policy, the each::labs NSFW Checker attains the highest reported Sexual-category F1 on UnsafeBench on both real-world and AI-generated images, shows no measurable degradation on generated content in its default mode, and exposes its strictness trade-off as three separated and individually measurable operating points. Performance outside the declared policy scope is poor and is reported as such, and the strictest mode over-blocks at a rate that a deployment must staff for.

Frequently asked questions

Is this an NSFW image to text model?

In the literal sense, yes. The input is an image and the output is text: a safety verdict together with a category drawn from none, nudity, sexual, sexual_minors, violence, hate and other. It does not caption the image or describe its contents. The text returned is a moderation decision, which is the signal a routing layer consumes.

Does the NSFW Checker work on AI-generated images?

UnsafeBench was constructed because conventional classifiers lose accuracy on generated content, in some cases by more than a tenth of an F1 point. In strict the measured difference between real-world and AI-generated images is -0.002 F1 across all eleven categories and +0.017 on the Sexual category. The intervals are wide, so the supported statement is that no gap was observed rather than that none exists.

Which mode should be used?

strict is the balanced default at 0.681 recall and 0.119 false positives on in-scope categories, and it holds the highest Sexual-category scores on the benchmark. moderate is appropriate where over-blocking is the costlier error, on adult or artistic surfaces. super_strict is appropriate where a miss is the costlier error, on child-safe and brand-safe placements, at a false-positive rate of 0.411.

Is it better than a frontier vision model?

On the Sexual category it holds the higher point estimate, 0.896 and 0.879 against GPT-4V's reported 0.866 and 0.827, but those values fall inside the reported confidence intervals, so the supported statement is a match at production latency. Against the purpose-built NSFW detectors the separation is unambiguous: 0.879 on AI-generated images against 0.678 and 0.596.

What does this benchmark not establish?

The false-positive rate a given deployment will observe. UnsafeBench's labels encode UnsafeBench's policy, which is broader than NSFW and differs from all three modes. These numbers compare classifiers on a fixed yardstick; the selected mode should be measured on a sample of the deployment's own traffic.

Eftal Yurtseven

Co-Founder & CEO

I’m the Co-Founder and CEO of each::labs, where I focus on building the infrastructure and tooling that help developers bring AI models into production. My background spans computer vision, augmented reality, machine learning, and software engineering, including building AR and visual AI products before moving deeper into generative AI. I’m particularly interested in the engineering challenges behind making powerful AI models fast, scalable, and practical for real-world products.

LinkedIn X

Close the Imagination Gap with each::labs