Content Moderation That Never Sees Your Private Messages

Most moderation systems claim they respect private messages. Almost none of them actually do, because the model still sees them. They run every message through a classifier first, then decide what to do with the result. The privacy boundary, if it exists at all, is a filter applied after the model has already seen the content. I built SifEdge, an open-source, on-device content moderation SDK, around a different rule: for private channels, the classifier never runs at all. Not "runs and gets discarded." Not "runs and gets logged locally only." Never runs. The check exits before any model, local or remote, ever sees the content. This post explains why that distinction matters, and why it's harder to get right than it sounds. The problem with "filter after" Picture a typical moderation pipeline. A message comes in, gets sent to a classifier, the classifier returns a toxicity score, and then some downstream logic decides whether the channel is public or private and whether to act on the score. The privacy boundary here is a business-logic decision, sitting downstream of the model. That has two consequences that matter more than they first appear to. First, it's a config flag, not a constraint. Someone can flip it, misconfigure it, or simply forget it exists in a new code path six months later. The classifier already saw the message either way, so the "privacy" only ever lived in what happens after inference, never in whether inference happened. Second, and more subtly: once content has passed through a model, it has already left the boundary you were trying to protect. Even if you don't log the score, even if you discard the result immediately, the content was still processed by something that isn't the two people in that private conversation. If that processing happens via an API call to a third-party service, which is how most moderation products work, the message left the device it was typed on before your privacy policy had a chance to matter. Making it structural SifEdge's private-channel gate works differently. Every check call takes a channelContext parameter. When it's 'private', the function returns immediately, before loading a model, before any inference, before any network call: async checkText(text, options = {}) { if (this.channelContext === 'private') { return { decision: 'ALLOWED', modality: 'text', skipped: true, reason: 'private-channel' }; } // ... classifier logic only runs past this point } Enter fullscreen mode Exit fullscreen mode That's the whole trick, and it's almost embarrassingly simple. The reason it's worth writing about isn't the code, it's the ordering. The gate is the first line, not a filter on the last line. There is no code path in the SDK where a private message reaches a classifier, because the function returns before the classifier is even referenced. The same discipline continues server-side. If you wire the SDK to a backend for logging and review queues, the backend independently rejects any decision payload tagged as a private-channel result: if (channelContext === 'private') { return res.status(400).json({ error: 'private-channel decisions must never be sent to the backend' }); } Enter fullscreen mode Exit fullscreen mode Two independent enforcement points, client and server, each one sufficient on its own to prevent private content from being scored or stored. If a developer misuses the client SDK, or wires up a custom integration that skips it, the server still refuses to accept the result. Why this is a harder promise than it sounds It would be easy to write a paragraph in a README that says "we don't moderate private messages" and leave it at that. Plenty of products do exactly this. The gap between a documentation promise and a structural guarantee is where most privacy claims quietly fail. A documentation promise can be true today and false after the next refactor. A structural guarantee, where the check happens before the classifier is even called, fails loudly: if someone tries to route private content through the pipeline anyway, they get a hard error, not silent non-compliance. That's the difference between "we don't do this" and "this cannot happen without someone deliberately removing the guard and shipping that change." It also means the privacy property doesn't depend on trusting every future contributor to remember why the check exists. The gate is load-bearing code, not a comment. What this doesn't solve I want to be honest about the limits here, because overclaiming is exactly the failure mode this whole design is trying to avoid. The gate protects against the SDK or its reference server scoring or storing private content. It does not, and cannot, prevent a platform from choosing not to mark a channel as private in the first place. channelContext is set by the integrator, not detected automatically. If a platform lies about which channels are private, or doesn't distinguish them at all, the gate has nothing to act on. This is a tool for platforms that want the boundary enforced correctly, not a guarantee that every platform using it will configure it honestly. It also doesn't solve moderation for private channels at all, by design. Some platforms have legal or safety reasons to want some signal even from private messages, for example detecting imminent harm. SifEdge takes the position that on-device automated scoring of private conversations isn't the right tool for that, and doesn't attempt it. That's a real trade-off, not a free win, and teams with different requirements should know that going in. The rest of the system, briefly The gate is one piece of a larger design, text, image, and audio moderation running locally via transformers.js, and a tiered warning system per modality instead of one shared counter. Full details are in the README. The code, tests, and full README are at github.com/MihaiMotoi/SifEdge, Apache 2.0.