EDGE, PLEASE.
01 / LOCAL AI, MODELS & MEMORY
Speech becomes a note. A note becomes something you can find. A small model helps you make sense of it—all close to the information itself.
A CONCEPTUAL WORKFLOW / NOT A LIVE PRODUCT DEMO
WHY THIS MATTERS
Personal AI gets interesting when it can work with the information you actually care about: a thought you spoke aloud, a document you saved, or a decision you want to revisit.
Running parts of that workflow locally creates opportunities for offline use and deliberate control over data movement. The whole experience still depends on model quality, memory, storage, software and the way permissions are designed.
THE MODEL STACK
Speech recognition, routing, retrieval and generation are different problems. Each deserves the right tool.
Whisper is a speech recognition model. whisper.cpp brings Whisper inference to a C/C++ runtime with CPU support, making local transcription a practical area to explore.
Accuracy and speed vary with the model, language, noise and target hardware.
whisper.cpp project ↗Cactus Compute’s Needle3 model card describes function calling and text embeddings. These are useful building blocks for selecting a defined tool or matching information locally.
A model’s suggestion still needs explicit application rules and permissions before an action happens.
Needle3 model card ↗Small instruction models can be explored for focused writing and summarization tasks. llama.cpp provides local inference across a range of hardware, including CPUs.
Model fit involves more than the parameter count: weights, context and runtime overhead all consume memory.
llama.cpp project ↗FROM FILES TO USEFUL CONTEXT
Retrieval lets an application find relevant material before asking a model to answer. It can combine text search with embeddings, which represent information for similarity matching.
Audio, transcripts and documents remain the reference for what happened.
Search selects useful context instead of sending an entire archive through a model.
An answer should make it easy to revisit its source and correct a misunderstanding.
OnCore’s direction brings private capture, local memory and user-controlled assistance into one personal system. These are areas of development; final capabilities and hardware will be shared as they are validated.
THE REAL TRADE-OFFS
Weights, working context and other services need room together. A file that fits on disk may still exceed the available RAM.
Measure the complete interaction: loading, transcription, retrieval and generating the answer you need.
A pocket-sized device has a different energy budget from a desktop. Useful work must fit its operating environment.
ONNX Runtime uses execution providers for CPUs and supported accelerators. The exact model and backend must work together.
ONNX Runtime documentation ↗MAKE THIS YOUR SIGNAL
Select your interests for future field notes and OnCore updates.
Choose my updates ↗NEXT / CHIPS, DEVICES & DEV KITS →