ORIS is a collection of independent experiments rather than one architecture. Choose a model to open its roadmap, versions, results and research notes.
The project where most of the early custom-model work happened: data preparation, tokenizer design, model internals, forward passes, numerical heads and difficult first attempts at exporting non-standard architectures to Hugging Face.
Two related training tracks: ORIS 660M kept the structural-recovery experiment; the Qwen 0.8B branch tested tokenizer replacement and training with part of the pretrained weights frozen.
Compact Polish BERT-style encoder developed for embeddings, fast local filtering, scoring and downstream fine-tuning.
One trained supernetwork, four deterministic compute profiles, and physically extractable models that reproduce their profile path exactly.
Planned ORIS vision-language model for understanding images and interacting with visual content in Polish.
Visual counterpart to ORIS BERT: a compact embedding-oriented encoder for image similarity, retrieval, scoring and dataset filtering.
A custom language-model experiment that became the main place to learn the complete model pipeline in practice — from data formatting and tokenizer training, through embeddings and transformer blocks, to forward passes, custom heads, decoding and the awkward reality of exporting non-standard solutions to Hugging Face. The numerical idea mattered, but so did learning how all the pieces actually fit together.
PolMATH did not begin as an adapter attached to an existing pretrained model. The transformer stack, tokenizer behaviour, numerical codec, embedding fusion, auxiliary losses, routing logic and numerical decoding path were implemented as one experimental architecture.
Text could contain a dedicated numerical position while the actual value travelled through a structured numerical representation. That representation could encode integer and fractional digits, lengths, decimal form and exponent structure rather than collapsing every number into one scalar or leaving it entirely to BPE tokenization.
This made the experiment less about “doing mathematics” and more about asking whether a language model benefits from separating what a number means from how that number happened to be written.
Written numbers could be normalized into a dedicated numerical position while preserving linguistic context around them.
Integer digits, fractional digits, notation flags and exponent structure were represented explicitly.
Numerical features were projected into the token representation and gated per position instead of affecting every token indiscriminately.
Categorical structure prediction and a bucket/regression path provided complementary ways of reconstructing values.
An auxiliary objective aligned text and numerical representations at number positions and penalized numerical leakage outside them.
The LM could first decide that a numerical token belongs in the sequence, then the numerical head could reconstruct the value.
PolMATH was the project where the author spent the most time taking the pipeline apart and putting it back together. The point of the notes below is not to dump the repository onto a webpage, but to expose the decisions that normally disappear behind AutoModel.from_pretrained().
The tokenizer branch trains a ByteLevel-BPE vocabulary while keeping [PAD], [BOS], [EOS], [NUM] and the ten digits available explicitly. A regex detects ordinary decimals, decimal commas, scientific notation and suffixes such as 1930s or 10th. The numeric magnitude is moved out of the text stream and the text receives [NUM] in its place.
pre, spans = self._preprocess_text_for_numbers(raw_text)
enc = self._tok.encode(pre)
# token stream: "Temperature is [NUM] degrees"
# side channel: {position_of_NUM: "23.5"}
This makes the tokenizer responsible for linguistic placement, while the numeric path can carry the value itself.
The categorical codec decomposes a number into notation flags, integer/fraction/exponent lengths, exponent sign and digit classes. In the current magnitude-only branch the leading value sign is intentionally separated from the numeric magnitude.
1939.0 → int=[1,9,3,9], frac=[0] 1.939e3 → int=[1], frac=[9,3,9], exp=[3] # different notation, recoverable structure
The normal token, position and token-type embeddings are built first. Numeric features are projected to hidden size. In the gated variant a small MLP sees both representations and produces a per-position gate; the numeric mask suppresses the channel everywhere except numerical positions.
proj = self.numeric_proj(numeric_features) g = sigmoid(self.gate_mlp(cat([x_base, proj]))) g = g * numeric_mask.unsqueeze(-1) x = x_base + g * proj
An auxiliary term experiments with aligning text and numeric representations where a number exists and penalizing numeric leakage elsewhere.
The language backbone is intentionally understandable: causal scaled dot-product self-attention, a triangular mask, pre-norm residual blocks and a GELU MLP. One default configuration in the branch uses a 32k vocabulary, 768 hidden width, 12 layers, 12 attention heads, 3072 intermediate width and 2048 positions.
h = self.ln1(x) x = x + self.attn(h, attn_mask=attention_mask) h2 = self.ln2(x) x = x + self.mlp(h2)
This was useful pedagogically: custom behaviour was isolated around numbers rather than hiding every component behind another abstraction.
The model can emit ordinary vocabulary logits and numerical predictions from the same hidden states. The categorical head predicts the components needed to reconstruct notation. An optional mixed head adds bucket classification and regression. A learned two-way router tests whether the model can choose which numerical path should dominate instead of hard-coding the choice.
logits = self.lm_head(x) pred_cat = self.numeric_head_cat(x) pred_mix = self.numeric_head_mix(x) gate = softmax(self.router(x), dim=-1)
The tokenizer script includes a direct sanity check on special-token IDs, digit IDs, encoded tokens, numeric positions and reconstruction. One test string deliberately mixes arithmetic, a negative decimal, a year-like form and an ordinal:
"8+9 is 17. Temperature is -23.5 degrees. In the 1930s, 10th place was okay."
Useful test families for this branch include round-trip encode/decode, notation-equivalence pairs, sign and suffix edge cases, scientific notation, numeric-mask leakage, gate behaviour, shape checks for every head, causal-mask checks, and reversed association prompts such as fact → year versus year → fact.
A large part of the learning curve was not the forward pass itself but packaging a custom configuration, tokenizer behaviour, extra numeric tensors and custom model outputs so that they behaved predictably outside the local training script. Early exports were inconsistent: a model can train locally and still fail the practical save → load → tokenize → forward → generate path expected by Hugging Face tooling.
The lesson from PolMATH was blunt: a custom architecture is not finished when loss.backward() works. Serialization, configuration fields, tokenizer special tokens, output schemas and reload tests are part of the architecture too.
The training pipeline reached the intended qualitative behaviours: matching number words to numeric values and back, deciding when a numerical position should appear despite ordinary text tokens dominating the corpus, and learning useful associations between values and facts.
One of the useful examples was the relation between 1939 and the outbreak of the Second World War. The interesting part was not only producing the number after the fact, but also retaining the association when the direction of the prompt was reversed.
Different written forms such as 1939, 1939.0 and 1.939e3 could be learned as related expressions of the same value rather than as unrelated text fragments.
PolMATH was not a general-purpose mathematical reasoner. Its value was narrower: it demonstrated that an explicit numerical path could coexist with ordinary language modelling and learn useful number-related behaviour.
Custom architecture and structured numerical representation experiments.
Numeric placement, notation equivalence, routing, generation and association tests.
The work is being cleaned up rather than simply continued as another checkpoint.
A cleaner experimental protocol and an English-language iteration are planned so the idea can be evaluated beyond a Polish-only setting.
A pair of practical model-training experiments. ORIS 660M is the structural-compression and recovery branch described below; the earlier Qwen 0.8B track focused on changing the tokenizer and continuing training while freezing part of the pretrained model.
The Qwen 0.8B experiment was a more conventional pretrained-model surgery than ORIS 660M. The working test replaced the tokenizer and continued training while keeping part of the inherited weights frozen. The practical question was how much of a pretrained model can be preserved when the interface between raw text and embeddings changes.
This branch was useful less as a final model and more as training practice: vocabulary replacement, embedding adaptation, deciding which modules should remain frozen, checkpoint compatibility and observing where transfer breaks when the tokenizer no longer matches the one used during pretraining.
The uploaded material documents the ORIS 660M branch in detail; the tokenizer-replacement / partial-freezing Qwen description here follows the project history supplied by the author, not a benchmark table from the provided files.
The lineage is Qwen 2.5 → Bielik-1.5B-v3 → ORIS 660M. The starting teacher had 32 transformer blocks. ORIS retained only 12 while preserving the teacher's hidden width, attention geometry, tokenizer, embeddings, feed-forward dimensions and LM head.
The selected blocks were 0, 1, 2, 3, 4, 21, 23, 24, 25, 29, 30 and 31. They were not chosen by simply keeping every second or third layer. Small ablations and calibration tests were used to identify less-sensitive regions and compare candidate 9–12 layer structures.
The extreme jump from teacher layer 4 to layer 21 means that downstream blocks receive representations unlike those they originally saw during pretraining. That mismatch is one of the central features of the experiment.
| Model | Parameters | Loss | Perplexity | Throughput | Peak VRAM |
|---|---|---|---|---|---|
| ORIS 660M step0 | 660M | 7.1613 | 1288.58 | 17.29k tok/s | 1.54 GiB |
| Bielik-1.5B | 1.596B | 2.2692 | 9.67 | 6.69k tok/s | 3.29 GiB |
Fixed evaluation on 128 sequences of length 1024.
The network was strongly damaged. The experiment became: can an inherited but structurally broken network reorganize itself through ordinary next-token training?
| Architecture | Initial validation loss | After ~2.03M training tokens |
|---|---|---|
| Selected 12-layer ORIS | 7.2737 | 4.3134 |
| Uniform 12-layer control | 8.9560 | 5.4880 |
| Selected 10-layer candidate | — | ~4.9100 |
These experiments were intentionally small and do not establish that the chosen structure is globally optimal. They do show that equal depth does not imply equal recoverability.
Main recovery used standard causal language modelling. Teacher/student KL was useful for screening candidate structures, but teacher logits were not the training objective during the main continuation run.
Continued training restored grammatical Polish, reasonable local continuations and recognizable semantic associations, but free generation remained unstable.
Observed issues included greedy repetition loops, topic drift, weak long-range coherence, hallucinated dates and numerical sequences, web/forum residue, metadata-like fragments, premature EOS at some checkpoints and high sensitivity to sampling strategy.
One of the strongest lessons was that lower loss did not translate monotonically into better generation. Syntax, factual accessibility, calibration, semantic control and free-generation stability could improve at different rates.
| Task | ORIS V2.5A |
|---|---|
| Belebele accuracy | 0.2256 |
| 8tags accuracy | 0.1757 |
| PoLeMo2 in accuracy | 0.4155 |
| PoLeMo2 out accuracy | 0.3684 |
| DYK binary F1 | 0.1268 |
| PSC binary F1 | 0.1627 |
| PPC accuracy | 0.4180 |
| CBD macro F1 | 0.1291 |
| KLEJ NER accuracy | 0.1025 |
| PolQA reranking | 0.5157 |
| Model | Params | Raw MC | Normalized MC | Continuation NLL |
|---|---|---|---|---|
| ORIS 660M | 660M | 0.750 | 0.500 | 3.3525 |
| Qra-1b | ~1.10B | 0.917 | 0.583 | 1.5715 |
Qra remained clearly stronger; the comparison tested whether ORIS had returned to a meaningful operating regime.
| Model | Parameters | Accuracy | Normalized accuracy | F1 |
|---|---|---|---|---|
| ORIS 660M | 660M | 0.399 | 0.386 | 0.0116 |
| APT3-1B-Base | ~1B | 0.330 | 0.300 | 0.3215 |
| Task | ORIS | APT3 |
|---|---|---|
| PoLeMo2 in accuracy | 0.43 | 0.34 |
| PoLeMo2 out accuracy | 0.33 | 0.38 |
| 8tags accuracy | 0.12 | 0.20 |
| Belebele accuracy | 0.23 | 0.25 |
| KLEJ NER accuracy | 0.31 | 0.05 |
| PolQA reranking accuracy | 0.62 | 0.39 |
| PPC accuracy | 0.44 | 0.42 |
| PSC accuracy | 0.68 | 0.32 |
ORIS could show surprisingly high accuracy on some tasks while binary F1 collapsed because of severe class bias. That is exactly why these results are diagnostic rather than a claim of general superiority over APT3.
An inherited 12-layer ORIS student was compared against an architecture-matched model initialized from scratch. Both had comparable structural capacity; only one inherited the teacher's pretrained parameter structure.
| ENTITY_ONLY subset | Before | After |
|---|---|---|
| Accuracy | 20.0% | 41.5% |
| Mean factual margin | -0.840 | +0.120 |
The result does not prove that complete symbolic facts remain intact inside individual weights. It does suggest that the inherited network begins recovery from a qualitatively different state than a random model with the same architecture.
A small generation probe captured the ambiguity: “Polska jest…” remained approximately correct, while “Kopernik był…” produced “Kopernik był w kosmosie.” The statement is false, but the astronomy/space semantic neighborhood survived.
Knowledge survives and remains directly usable.
The semantic region survives while precise retrieval fails.
Useful structure may survive but no longer be reachable through the altered path.
The information genuinely has to be reacquired during continued training.
Later generations repeatedly exposed artifacts from web-derived training data: forum fragments, metadata, SEO text, navigation elements, date-heavy sequences, list structures, transcripts, document templates and poorly separated topic boundaries.
The newer pipeline therefore separates linguistic quality, structural quality, corruption, noise, difficulty, domain, information density and knowledge value rather than compressing all of them into one score.
A difficult scientific or legal document can have high perplexity and many rare tokens while still carrying high knowledge value. A fluent generic paragraph can look clean while contributing almost none.
Stable IDs, provenance, validation, sharding and raw preservation.
Encoding repair, paragraph preservation, segmentation and conservative local deduplication.
Language confidence, coherence, repetition, information density, rare-token statistics and perplexity.
Separate estimates for quality, noise, knowledge value and reconstruction decisions.
Domain hierarchies, subdomains and knowledge flags.
Global exact/fuzzy deduplication, redundancy control and later curriculum balancing.
The working hypothesis is not that one bad example damages a foundation model. The concern is cumulative: an unstable model repeatedly sees similar defects, small representation errors reinforce each other, and later data is interpreted through an already-shifted internal state.
In that view, data ordering can matter as much as data inclusion.
Coherent, lower-risk data intended to restore reliable autoregressive behaviour.
Broader domains, higher information density and more difficult material after stabilization.
Harder distributions introduced under tighter control.
This curriculum is proposed, not completed.
ORIS is an independent, self-funded project developed primarily on privately owned compute. A substantial portion of pruning, validation, training and evaluation was performed locally, including on an RTX 5060 Ti.
The existing experiments suggest that a severely depth-reduced Polish model can recover useful language-model behaviour. The harder questions are now what exactly survives structural reduction, what is genuinely relearned, whether inherited knowledge can be distinguished from reacquired knowledge, and whether recovery can be controlled through data ordering at 10B–100B+ token scale.
The experiment clarified the next bottleneck. Work shifted toward data infrastructure, evaluation methodology, knowledge-recovery controls and additional compute before committing the 660M branch to a much larger run.
ORIS BERT is the continuation of the model originally developed under the working name ORIS Bert Small C. It uses BERT-style masked-language pretraining and is treated primarily as a compact encoder / embedding model, not a stock scaled-down BERT.
Dataset preparation for ORIS 660M became a throughput bottleneck, so a smaller encoder was built specifically for local filtering, categorization and scoring on NVIDIA Blackwell hardware.
What began as an internal pipeline tool became a compact Polish encoder intended primarily as a backbone for task-specific fine-tuning.
Raw mean-pooled embeddings are strongly anisotropic and are not recommended for zero-shot semantic search without additional contrastive or task-specific fine-tuning.
| Property | Value |
|---|---|
| Model type | Custom Transformer encoder |
| Hidden size | 384 |
| Token embedding size | 128 |
| Attention heads | 6 |
| Context length | 1024 |
| Attention layout | 256, 256, 1024, 256, 256, 256 |
| Normalization | RMSNorm |
| Objective | Masked Language Modeling |
| Initialization | From random initialization |
Five of six layers use local 256-token attention. The third layer performs full 1024-token attention and acts as the main global mixing layer.
The vocabulary is intentionally large, but the token embedding dimension is only 128 while the encoder hidden size is 384. This factorization keeps the 128K vocabulary from dominating the parameter budget.
During MLM pretraining, vocabulary logits are calculated only for masked positions. Those hidden states are projected from 384 dimensions back into the 128-dimensional token space and scored using the tied token-embedding matrix.
| Setting | Value |
|---|---|
| Sequence length | 1024 |
| Micro-batch size | 16 |
| Gradient accumulation | 8 |
| Tokens / optimizer update | 131,072 |
| Optimizer updates | 61,036 |
| Peak learning rate | 3e-4 |
| Mask probability | 15% |
| Precision | BF16 autocast |
| GPU | RTX 5060 Ti 16GB |
| Training time | ~13.44 h |
| Average throughput | ~165.4K input tok/s |
Training data was primarily Polish MADLAD with an auxiliary mixture containing Wikipedia, OpenSubtitles PL, balanced NKJP and Polish legal/judicial text. The auxiliary pool represented roughly 15% of generated training sequences.
The tokenizer is a custom 128K BPE with NFKC normalization and Metaspace pre-tokenization. It is Polish-oriented but includes a broad Unicode alphabet for noisy web text.
| Model | Mean cosine for unrelated texts |
|---|---|
| ORIS BERT | ~0.987 |
| HerBERT | ~0.90 |
| PolDense | ~0.26 |
Internal diagnostic only; not a general encoder-quality benchmark.
| Task | Metric | ORIS BERT | PolBERTa base |
|---|---|---|---|
| NKJP-NER | Macro-F1 | 75.52 | 84.36 |
| CDSC-E | Accuracy | 91.30 | 91.00 |
| CDSC-R | Spearman | 88.18 | 88.97 |
| CBD | F1(+) | 50.24 | 43.75 |
| PolEmo2.0-IN | Accuracy | 83.33 | 85.32 |
| PolEmo2.0-OUT | Accuracy | 65.59 | 63.77 |
| DYK | F1(+) | 37.86 | 46.31 |
| PSC | Macro-F1 | 57.28 | 85.87 |
| AR | MAE ↓ | 0.5929 | 0.5753 |
Local evaluation using the same fixed procedure for ORIS BERT and PolBERTa base; not an official KLEJ leaderboard submission.
| Metric | mmBERT-base | ORIS BERT |
|---|---|---|
| Decision Macro-F1 | 0.4334 | 0.5015 |
| Decision accuracy | 0.5185 | 0.6296 |
| Training time | 583.2 s | 124.4 s |
| Peak VRAM | 5.83 GiB | 0.52 GiB |
| Metric | mmBERT-base | ORIS BERT |
|---|---|---|
| Full pipeline time | 21.732 s | 4.408 s |
| Documents / second | 11.78 | 58.07 |
| Mean latency / document | 84.89 ms | 17.22 ms |
| Peak VRAM | 1.806 GiB | 0.252 GiB |
This is a task-specific pipeline result, not a claim that Small C is universally superior to larger encoders.
| Setting | ORIS BERT | mmBERT-small | Advantage |
|---|---|---|---|
| Batch 1, 128 tokens | 3.828 ms | 17.153 ms | 4.48× |
| Batch 1, 1024 tokens | 3.940 ms | 17.229 ms | 4.37× |
| Batch 8, 1024 tokens | 1.25M tok/s | 159K tok/s | 7.85× |
Different tokenizers mean cross-model tokens/s should be interpreted carefully.
Raw mean-pooled sentence representations have poor cosine-space separation. Retrieval, ranking and sentence similarity need additional fine-tuning. Larger Polish encoders remain stronger on several benchmarks, and five local-attention layers mean cross-window communication relies heavily on one global mixing layer.
The model is therefore best understood as a compact MLM-pretrained backbone for supervised tasks, filtering and feature extraction — not as a universal zero-shot embedding model.
Built to make local filtering and scoring fast enough to matter.
8.00B-token from-scratch pretraining completed.
Used as a practical encoder backbone while downstream and representation-space behaviour are evaluated.
Further architecture, training and evaluation work may turn the concept-stage checkpoint into a fuller encoder family.
One shot, one compute, many possibilities. One jointly trained model exposes four deterministic compute profiles and can later be physically reduced into smaller standalone models without changing what a given profile computes.
Vyuhu does not learn a router that decides which experts receive each token. A profile is selected before the forward pass. Mandatory anchor blocks are always present; optional heavy compute is executed according to a fixed profile schedule. Cheap transfer paths and low-rank controllers keep representations compatible when compute is skipped.
The result is one training lineage with several explicit operating points: Vasudeva, Sankarshana, Pradyumna and Aniruddha. The fifth object in “4 in one? Nah, 5.” is the complete shared supernetwork from which those paths come.
The largest deterministic path. Nothing optional is withheld.
A reduced path intended to land near the ~175M class in the first ~280M family run.
A smaller operating point targeted near the ~125M class.
The smallest deterministic path, targeted near ~100M in the first full family run.
Train the shared system once. Select a deterministic compute level at inference — or extract that level into its own standalone model later.
The validation run was not treated as a final quality leaderboard. Its purpose was narrower and more important: determine whether one jointly trained network could preserve a stable compute→quality ordering across deterministic profiles, and whether a selected profile could be physically removed from the supernetwork without silently changing its function.
At roughly 0.832B evaluated tokens, quality remained monotonic with compute: the full Vasudeva path was best, followed by Sankarshana, Pradyumna and Aniruddha. No profile inversion appeared in the final evaluation.
| Profile | Schedule | Physical extracted size | Eval loss | Perplexity |
|---|---|---|---|---|
| Vasudeva | [4, 4, 4] | 125.721M | 3.2705 | 26.32 |
| Sankarshana | [1, 2, 1] | 75.602M | 3.3174 | 27.59 |
| Pradyumna | [0, 1, 0] | 56.723M | 3.4138 | 30.38 |
| Aniruddha | [0, 0, 0] | 50.430M | 3.5446 | 34.63 |
The extracted model reproduces the selected profile exactly: Δlogit = 0 in the extraction test. A smaller extracted artifact can therefore be deployed without an additional quality loss caused by the act of removing inactive structure. The profiles themselves still have different quality because they intentionally use different amounts of compute.
The next target is deliberately smaller than the 500M engineering scale test. Width is reduced while the proven four-profile topology is retained, with the middle profiles raised by one heavy block so the family lands near four useful deployment classes rather than clustering too tightly at the bottom.
| Profile | Planned schedule | Approx. family class | Role |
|---|---|---|---|
| Vasudeva | [4, 4, 4] | ~275–285M | full model |
| Sankarshana | [1, 3, 1] | ~175M | balanced middle-high path |
| Pradyumna | [0, 2, 0] | ~125M | compact middle-low path |
| Aniruddha | [0, 0, 0] | ~100M | minimum anchor path |
Sizes are current design targets for the first ~280M run, not yet final checkpoint counts.
32K corpus-specific byte-level tokenizer prepared for the Polish training corpus and exact Unicode round-trip behaviour.
Hardware-friendly width with 64-dimensional attention heads.
18 query heads / 6 KV heads in the current ~280M design. Optional heavy blocks use the elastic mixer rather than full global attention.
Dense SwiGLU channel capacity retained across anchor and elastic compute.
Hybrid optimizer path: Muon for selected 2D hidden matrices, fused AdamW for the remaining parameter groups.
Controller/bypass details, training mixture and a few run-level knobs stay intentionally unpublished until the experiment is complete.
A useful conventional reference is the published ~275M configuration shown below. It is not presented as a controlled Vyuhu benchmark: the architectures, optimizer path, tokenizer and training data differ. The point is simply to make the scale tangible before the first ~280M Vyuhu run exists.
| Hyperparameter | Conventional ~275M reference | Vyuhu ~280M target |
|---|---|---|
| Model parameters | 275M | ~275–285M supernetwork target |
| Sequence length | 1024 | 1024 |
| Vocabulary | 31,980 | 32,000 · VYUHU32k |
| Transformer layers | 32 | 4 mandatory GQA anchors + elastic staged compute |
| Attention heads | 16 MHA | 18 Q / 6 KV in GQA anchors |
| Head dimension | 64 | 64 |
| Model width | 768 | 1152 |
| Intermediate size | 2048 | 3584 |
| Positional encoding | RoPE | RoPE |
| Activation | SwiGLU | SwiGLU |
| Normalization | RMSNorm · ε 1e-6 | RMSNorm · ε 1e-6 |
| Dropout / bias | 0.0 / no | 0.0 / no |
| Optimizer | AdamW | Muon + fused AdamW |
| Exact MB / GA / LR schedule | 13 / 40 / 4e-4 → 2e-5 | :) |
A conventional dense 275M checkpoint is one operating point. Vyuhu is being designed so a single training run can yield a ~280M full path and useful ~175M, ~125M and ~100M deterministic descendants — with exact extraction rather than post-hoc pruning.
The ~500M branch was used as an engineering stress test for memory, optimizer geometry, profiling and custom-kernel work on a single RTX 5060 Ti 16 GB. The current training path reached roughly 11.8k tokens/s at sequence length 1024 with MB 8 / GA 16.
This is a throughput smoke test, not a quality claim for a finished 500M checkpoint. It demonstrated that the architecture, hybrid optimizer and current runtime can execute at this scale on the target desktop GPU.
Vasudeva: The form that answers when nothing needs to be left behind.
Sankarshana: The same answer, after deciding some of itself can remain silent.
Pradyumna: What appears when less is allowed to be enough.
Aniruddha: The version that still reaches the answer after forgetting how much it was supposed to carry.
Vasudeva: I.
Sankarshana: I.
Pradyumna: I.
Aniruddha: I.
Supernetwork: You trained me once.
Vyuhu · active architecture research · first ~280M full run in preparation
A planned ORIS vision-language model focused on understanding images and interacting with visual content in Polish.
Vidar-VL is intended as the ORIS vision-language branch for image understanding, visual question answering, OCR-oriented interaction and broader multimodal reasoning.
Further technical details will be published when the project reaches its first experimental release.
A planned compact visual embedding model — the visual counterpart to ORIS BERT — for similarity, retrieval, scoring, analysis and filtering.
ORIS Vision is intended as the visual counterpart to ORIS BERT: both are compact encoder-style models aimed at representations rather than conversational generation. Planned uses include image embeddings, similarity, retrieval, scoring and image filtering.
Architecture, training sources and implementation details are intentionally not disclosed at this stage.