Feedback request: Prototype of optional on-device Copilot for CryptPad

flisowna

Hi Cryptpad Team

As mentioned already in conversations, we'd like to describe here once more our project idea: we would like to develop an optional, fully client-side "Copilot" module for CryptPad as a working prototype in a forked repository. The goal is to add lightweight text-assistance features - entirely on-device, without changing the existing security or encryption model.

We’d love to get technical feedback and guidance from the CryptPad core team.

Motivation

Modern “cloud copilots” are powerful but not usable in E2E-encrypted, privacy-focused platforms like CryptPad. At the same time, WebGPU inference for small language models (LLMs) has become fast enough to be practical on end-user devices.

Our goal: developing a usable prototype for summarization and rewriting directly into CryptPad’s document app - completely offline, no data transfer (besides the initial loading of the LLM itself).

This would make AI assistance available to NGOs, educators, public institutions and all other users that cannot use cloud-based AI tools for compliance reasons or data privacy concerns.

Core idea & architecture

High Level Target Architecture Diagram
Image description

Data/Controll Flow Diagram
Image description

Description of implementation approach

Inference: runs in a same-origin Dedicated Web Worker (WebGPU accelerated)
Models: small, instruction-tuned decoder LLMs (e.g. Phi-4 Mini or TinyLlama-1.1B)
Model assets: self-hosted by the instance (connect-src 'self'), cached in OPFS/IndexedDB, progressive download with resume
UI integration: editor panel inside Inner (Docs), communication via Inner -> Outer -> Worker -> Outer -> Inner message flow
Security: strict CSP; no third‑party/cross‑origin requests; model assets served from same origin; no telemetry; prompts/responses in‑memory
Feature flag: disabled by default; optional module that admins can enable locally (zero-impact when off)
Undo-safe insertion: via existing editor API (not direct DOM manipulation)

Questions

What we’d love from the CryptPad team are your comments/thoughts regarding:

Do you see any potential conflicts with CryptPad’s CSP or E2E model which might not be easy to spot? To our understanding there shouldn’t be any since the Copilot panel would live inside the Inner iframe (Docs UI), all LLM requests are brokered by the Outer to a same-origin Dedicated Worker. Model assets would be self-hosted and cached in OPFS. Prompts/responses would remain in memory and no plaintext crosses beyond the established Outer/Inner trust boundary.
Are there any other technical aspects that could make this approach impractical?

Thanks for taking the time to review and share your thoughts.

ldubost

Hi @spaudanjo,

Thanks for your proposal of contribution. It's nice to see some people try to think about what could make sense for CryptPad in terms of future features.

When it comes to AI, we are of course very afraid of any leaks that could happen with data that is protected by CryptPad, as AI system are currently mostly running on external Cloud platform with zero end to end encryption and also that additional risks exist if AI system include features to call the internet using the data transmitted to AI.

Now there are possibilities to not allow this to happen, and one of them is your proposal to run AI fully client side and fully control the features provided to make sure there are not data leaks. Now it's difficult for us given our time to dive in the proposal deep enough to validate the security approach. For this reason we believe it's very important to keep this feature as a plugin that can be evaluated separately than CryptPad. Now your approach is something that we would be able to support as a way to provide AI.

Another aspect we believe is very important is frugality. We are as a team very concerned of technology that consumes power (or water) for little benefits. Therefore we would also support solutions that focus on the most important AI features while looking specifically on how to reduce the power used to provide the features. There are such features such as translation and summarization that are interesting on this aspect. We would also be very interested if users would be informed with an evaluation of the power that was used by a task they perform.

With those aspects taken into account, we could support you to build a plugin to the CryptPad platform (as our time permits) so that users that wish it, could run a "safe and responsible AI system". This is something we as a team have unfortunately not the time to work on, while we can see that some users could want it to switch to a privacy respectful platform such as CryptPad.

Ludovic