Collections and chains

Read, filter, sort, join and write JSON documents from a synchronous chain of calls on a collection.

Docs menu

A collection (db.collection(name), or db.<name>) is also a class: its rows are instances of it. Every example on this page runs as shown; a trailing // value comment is what the line answers. See Chain reference for every chain method's arguments, answer, errors and example, the filter object's operators, and how a chain runs.

Quick start

const Users = db.collection("users");
const Posts = db.collection("posts");
Users.hasMany("posts", Posts, "authorId");

const ann = new Users({ name: "Ann", age: 31 });
ann.save();
Posts.insert({ authorId: ann._id, title: "Hello", likes: 3 });
Posts.insert({ authorId: ann._id, title: "Again", likes: 9 });

Users.gte("age", 18).with("posts").first().posts.length;     // 2
Posts.eq("authorId", ann._id).gt("likes", 5).pluck("title"); // ["Again"]
Posts.updateWhere({ authorId: ann._id }, { $inc: { likes: 1 } }); // 2
ann.age = 32;
ann.save();
ann._version;                                                // 2
Users.get(ann._id).age;                                      // 32

Collections as types

const People = db.collection("people");
const bo = new People({ name: "Bo", tags: ["a"] });
bo.save();
bo._version;                                   // 1
typeof bo._id;                                 // "string"
People({ name: "Cy" }).save();
const di = People.create({ name: "Di" });
di._version;                                   // 1
bo instanceof People;                          // true
People.get(bo._id) instanceof People;          // true
db.people === People;                          // true
People.count();                                // 3

A collection is db.collection(name) or db.<name>, the same object each time. _id is a string the database makes. _version counts the writes of a document (1 on insert). A Date is stored as its ISO string, undefined fields are left out. Collection writes join db.transaction: they commit with it, and a rollback takes them back with the SQL writes (sql.md).

Four names are reserved on every document: _id, _createdAt, _updatedAt and _version. On insert, insertMany and create the collection assigns _id, so a document that names _id is refused, never silently dropped: an ID_RESERVED Error ("insert: _id is assigned by the collection; use put(doc) to choose an id", and insertMany[i] naming the row), with nothing written. The other three are the system's stamps and are ignored when a document names them. To choose an id, use put(doc), putMany(docs) or upsert('_id', doc): they take doc._id as the id the document is stored under.

Reading

const Articles = db.collection("articles");
const ids = Articles.insertMany([
  { title: "a", n: 1, tag: "x" },
  { title: "b", n: 2, tag: "y" },
  { title: "c", n: 3, tag: "x" },
]);
Articles.get(ids[0]).title;                    // "a"
Articles.get("zzzzzzzz");                      // null
Articles.eq("tag", "x").all().length;          // 2
Articles.eq("tag", "x").first().title;         // "a"
Articles.eq("tag", "x").count();               // 2
Articles.eq("tag", "x").ids();                 // [ids[0], ids[2]]
Articles.eq("tag", "x").pluck("title");        // ["a", "c"]
Articles.eq("tag", "x").last().title;          // "c"
Articles.sort("n", -1).last().title;           // "a"
Articles.eq("tag", "x").exists();              // true
Articles.eq("tag", "z").exists();              // false
Articles.all().length;                         // 3
Articles.first().title;                        // "a"
Articles.last().title;                         // "c"
Articles.count();                              // 3
Articles.ids().length;                         // 3
Articles.pluck("title");                       // ["a", "b", "c"]
Articles.find({ tag: "y" })[0].title;          // "b"
Articles.findOne({ tag: "y" }).title;          // "b"
Articles.count({ tag: "x" });                  // 2
Articles.exists({ tag: "z" });                 // false
Call Answers
get(id), findById(id) the row, or null
live(id) the document as a live row: each assignment to a field is written at once, no save(); null when absent
db.users[id], id in db.users, db.users[id] = doc, delete db.users[id] live(id) for an integer id up to 2^32 - 2, whether it is stored, a replace of a stored document (a missing id throws KEY_NOT_FOUND), a delete. A synced collection's ids are wider: use live(id) there
for (const doc of db.users) every document as a live row, in id order, read 64 at a time
class User extends db.users.Model { ... }, db.users.use(User) binds the class: every live row of the collection is then a User (live(id), db.users[id], for...of), with the class's getters and methods; one class per collection, a later binding replaces it
new User(doc) inserts doc once and answers it as a live User: an assignment in the constructor after super(doc) is a write
User.live(id), User.where(...), for (const u of User) the collection's live(id), chains whose all() and first() answer User instances, iteration; find and findById still answer plain rows
the first use of User use(User), new User(doc) or a static called on User binds it; rows made before the binding keep the prototype they were made with
a held live row a snapshot of its document, taken when the row was read; every read of a field is the row's own slot. A write through it lands only while the document is as the row read it (or as the row's own last write left it). After another writer changed the document, the write throws an Error with code 0x4012 (ROW_STALE) naming the collection and the field: nothing is written, and the row then holds the document as it is, so row.n = row.n + 1 read again and retried computes from the new value. No update is lost. Fields another writer added since the row was read are read through a new live(id)
where to declare User at module scope, once: a class declared inside the handler is a new class on every request, so every request binds again and makes one live Structure for each row shape it reads
find(filter, { sort, limit, skip, select }), findOne(filter) rows from a filter object ({ n: { $gt: 1 } }); no filter is every row. sort takes one key or several ({ a: 1, b: -1 } or [["a", 1], ["b", -1]]); with select: [fields] the rows are plain objects of those fields
count(filter), exists(filter) a number, a boolean (a filter object, never a field name)
all() the rows of a chain
first(), last() the first or last row in the chain's order, or null
firstOrFail() the first row; throws a NotFoundError when the chain matches nothing
value(field) one field of the first row, or undefined when nothing matches or the row has no such field
exists(), missing() true when a row matches, and its negation (with a field name, exists(f) is a condition, see Filters)
count(), ids(), pluck(field), columns() a number, id strings, one field of each row, an object of columns

A chain starts at the first condition (Articles.eq(...)) or at the collection itself, and ends at a terminal. all, first, firstOrFail, last, value, missing, count, ids, pluck, sum, avg, min, max, values, distinct(field) and facets also run directly on the collection: first and last read the ends of the collection, count its counter. last() of a sorted chain walks the sort in reverse and stops at the first row (among rows that tie on the sort value it answers the lowest id); with no sort it is the last row in id order. On the collection exists() takes a filter object (exists(filter)); with none it answers whether the collection holds a document.

with, withCount, whereHas and whereNone also start directly at a collection. So do page and pages: Articles.page({ size: 20 }) answers the first page without adding a condition; Articles.pages({ size: 20 }) iterates every page. Collection sort and sort_by accept either a field and direction or an array of keys, just like the chain methods.

Filters

const Products = db.collection("products");
Products.insertMany([
  { sku: "a1", price: 5, note: "x" },
  { sku: "a2", price: 15 },
  { sku: "b1", price: 25 },
]);
Products.eq("sku", "a1").count();                   // 1
Products.neq("sku", "a1").count();                  // 2
Products.gt("price", 5).count();                    // 2
Products.gte("price", 5).count();                   // 3
Products.lt("price", 25).count();                   // 2
Products.lte("price", 15).count();                  // 2
Products.in("sku", ["a1", "b1"]).count();           // 2
Products.nin("sku", ["a1", "b1"]).count();          // 1
Products.between("price", 5, 15).count();           // 2
Products.startsWith("sku", "a").count();            // 2
Products.gte("price", 0).like("sku", "%1").count(); // 2  (a text under 3 bytes needs another keyed condition)
Products.gte("price", 0).ilike("sku", "A%").count(); // 2
Products.gt("price", 0).exists("note").count();     // 1
Products.gt("price", 0).exists("note", false).count(); // 2
Products.and(q => q.gt("price", 5), q => q.lt("price", 25)).count(); // 1
Products.or(q => q.eq("sku", "a1"), q => q.eq("sku", "b1")).count(); // 2
Products.not(q => q.eq("sku", "a1")).count();       // 2
Products.where({ price: { $gt: 5, $lt: 25 } }).count(); // 1  (a filter object, as find takes it, ANDed with the chain)
Method Matches Note
eq(f, v), neq(f, v) (ne) equal, not equal a document without the field matches neither
gt, gte (ge), lt, lte (le) comparison number or string
in(f, [v]), nin(f, [v]) one of, none of
between(f, lo, hi) lo <= f <= hi both included; the executor runs it as gte(f, lo) and lte(f, hi) (one range of the keys); a null or missing field never matches; a null bound is refused
startsWith(f, s) text prefix
like(f, p), ilike(f, p) % any run, _ one character, \ escapes the next character the executor runs them, no SQL: like is case-sensitive, ilike folds ASCII letters only (as icontains); a field that is not a string never matches. abc% is startsWith, %abc endsWith, %abc% contains, abc eq; any other pattern is tested on each candidate behind a startsWith of its literal prefix, and its candidates come from the trigram keys of its longest literal segment when that is 3 bytes or more (a pattern with no such segment needs another keyed condition, or is refused: see the text conditions below). Wherever the chain runs through SQL (a subquery, a compound, a join the document route declines) it is refused, as contains and regex are; like of a joined field keeps SQL's rules
exists(f, present = true) the field is present (absent) chain only
and(...fns), or(...fns), not(fn) q => q.eq(...) builders sort, limit, join ... are refused inside
match(f, words, mode) text field search "all" (default) or "any"; field declared with text(f)

Values are null, boolean, number, string, Date or db.arg(n) (see "Kept chains"). _id takes eq, neq, in and nin only.

between of a field of the collection is gte and lte of that field, ANDed, so it runs on the same route as they do (explain() shows the executor's route, not relational), and inside not(...) it negates like them: a document with a null, missing or other-typed field is not in the range, so not keeps it. A joined field's between and a having between still run through the SQL engine with SQL's rules.

Null and missing

A leaf matches a document only when the field holds a value that satisfies it. The one leaf that matches the absence of a value is eq(f, null): it matches the documents whose field is null and the documents that do not have the field at all, the same rows as exists(f, false). neq(f, null) and exists(f) match the documents holding a value. sort puts the documents with no value (null or missing) last, ascending and descending, after the numbers and the strings, equal values in row order.

eq(f, null) and exists(f, false) of a top-level field are answered from the field's absent keys: one key per document whose field is null or missing, built by the first query that asks for them (from the documents already stored) and kept by every insert, update and delete after that, so the run reads those keys and no record. A path inside another field (o.y) has no absent key and is answered from the documents. Other collections, and a database written before the keys existed, simply have none until a query asks.

Nested paths

A leaf or a sort key may name a member of a document held in a field with a dotted path: eq("address.city", "Paris"), gt("stats.views", 10), in, between, startsWith, ieq, like, contains, exists("address.geo", false) and sort("profile.age") (a second key too). Rules:

  • A name is split on .. A top-level field whose whole name is that very text, if the collection's documents hold one, is that field and wins over the path (decided when the chain is run).
  • A path whose member, or any step on the way, is missing or null, or a step that is not a document (a number, a string), reads as a missing value: exists(path, false) and eq(path, null) match it, and every other leaf does not. A document at the end of the path is a value that equals and orders with nothing (exists(path) matches it; a sort over it is refused with cannot order).
  • Arrays are not traversed. A document holding an array on the path (at a step or at its end: { a: [{ b: 1 }] }, { a: { b: [1, 2] } }) is never matched by a leaf over a.b, whatever the leaf. Once an array has been keyed under the path (the path's keys are built by the first eq, neq, in, nin, startsWith, range, sort or ieq over it, and notice every array in the documents) every chain over the path is refused with a nested path does not traverse arrays; before that (a range or exists with the document route off, which reads no keys) the records are tested and the arrays simply never match. Read an array field with containsAll, size or elemMatch, which take a top-level name.
  • The path's keys are built on the first filter that needs them, exactly as a top-level field's, and kept by every insert, update (replacing the parent document, a scalar or no parent), unset and delete. exists(path, false) and eq(path, null) have no absent keys, and ne, nin, not, exists(path), regex and the other leaves have no key of any kind: beside a keyed condition (eq, in, a range, startsWith, ieq, a text needle of 3 or more characters) they are tested on that condition's candidates; a filter over a path with no keyed condition is answered by one pass over the collection's documents, each record read once and the pass ending at the limit. A dotted key in $set, $unset, $inc, $mul, $min or $max writes the member (see Update operators): { $set: { "address.city": "Oslo" } } sets member city of address.
  • select and pluck take a path and read it in place from each record (no keys are read for it): select('name', 'address.city') answers { name, address: { city } } (members under one head share its object; a name selected whole holds its members), pluck('address.city') and value('address.city') the values. A member the document lacks (a missing step, a scalar or an array on the path) is absent from a select row and undefined in a pluck, as a missing top-level field is; a stored null is null. Only a plain chain answers a select path: with join, groupBy, distinct, a compound or columns() it is refused with a nested field path (a.b) is read by a plain chain.
  • groupBy, aggregates, joins, the array leaves (containsAll, size, elemMatch), eqField and the other two-field forms, live queries, segments and the HTTP chain take top-level fields only: a path there is refused with names a nested field. With a path in the chain, a cursor page, a search leaf, a join or a relational method is refused too.

Negations

const Tasks = db.collection("f1tasks");
Tasks.insertMany([{ n: 1, state: "open" }, { n: 2, state: "done" }, { n: 3, state: null }, { n: 4 }]);
Tasks.ne("state", "open").pluck("n");                             // [2]
Tasks.nin("state", ["open", "done"]).pluck("n");                  // []
Tasks.not(q => q.eq("state", "open")).pluck("n");                 // [2, 3, 4]
Tasks.gte("n", 0).exists("state", false).pluck("n");              // [3, 4]
Tasks.eq("state", "done").not(q => q.eq("n", 9)).pluck("n");      // [2]

neq, nin and not answer the live documents minus the ones the condition selects (a bitmap and-not, never a row walk): neq and nin keep only the documents that hold a value, not(fn) every document the conditions do not select (a missing field included), exists(f, false) the documents without a value.

Array conditions

const Posts = db.collection("f1posts");
Posts.insertMany([
  { n: 1, tags: ["js", "db", "perf"], lines: [{ sku: "a", qty: 1 }, { sku: "b", qty: 5 }] },
  { n: 2, tags: ["js"], lines: [{ sku: "a", qty: 3 }] },
  { n: 3, tags: [], lines: [] },
  { n: 4, tags: "js" },
]);
Posts.containsAll("tags", ["js", "db"]).pluck("n");               // [1]
Posts.containsAny("tags", ["perf", "js"]).pluck("n");             // [1, 2]
Posts.size("tags", 0).pluck("n");                                 // [3]
Posts.sizeGt("tags", 1).pluck("n");                               // [1]
Posts.sizeLt("tags", 2).pluck("n");                               // [2, 3]
Posts.elemMatch("lines", { sku: "a", qty: { $gte: 3 } }).pluck("n"); // [2]
Posts.elemMatch("lines", { qty: { $gt: 4 } }).pluck("n");         // [1]
Method Matches Note
containsAll(f, [v]), containsAny(f, [v]) an array holding every, any of the values elements compare like eq (1 is not "1"); read from the field's element keys, then confirmed on the record; an empty list matches every array (containsAll) or none (containsAny)
size(f, n), sizeGt(f, n), sizeLt(f, n) the array's length tested on the record of each candidate
elemMatch(f, filter) an array of objects with an element satisfying every property of filter a value is an equality; an object of $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists, $startsWith, $endsWith, $contains, $icontains; tested on the record of each candidate

A document whose field is not an array (a string, a number, null, missing) matches none of them.

Text conditions

const Words = db.collection("f1words");
Words.insertMany([{ w: "Hello World" }, { w: "hello" }, { w: "WORLD" }, { w: "x.y" }, { w: 12 }]);
Words.endsWith("w", "World").pluck("w");                          // ["Hello World"]
Words.contains("w", "ell").pluck("w");                            // ["Hello World", "hello"]
Words.icontains("w", "world").pluck("w");                         // ["Hello World", "WORLD"]
Words.regex("w", /^h/i).pluck("w");                               // ["Hello World", "hello"]
Words.regex("w", "o$").pluck("w");                                // ["hello"]
Method Matches Note
endsWith(f, s), contains(f, s) a string ending with, holding s case sensitive; the empty string is in every string
icontains(f, s) holding s ASCII letters compared without case
regex(f, re) the pattern is found in the string re is a string or a RegExp, ECMAScript grammar; only the i flag (another flag is UNSUPPORTED_OPERATION); a pattern that does not compile is INVALID_QUERY_SYNTAX

endsWith, contains, icontains and the like / ilike patterns that are not a prefix are answered from the trigram keys of the field (every 3-byte window of the text, ASCII letters without case), built by the first query that needs them and kept current by every write after. Each candidate is then tested exactly on its record, under the leaf's own case rule. A needle no document can hold answers empty at once. Texts are keyed in full, however long, so a match is never missed; a multi-byte character matches the same way in the text and in the needle. explain() shows the keyed route (driver.kind keyedSets, driver.keys trigram).

A needle under 3 bytes has no key (unless the field is declared keyword). Beside another keyed condition (eq, in, a range, startsWith) it is a per-candidate test on that condition's rows. Alone (or in an or with only unkeyed company) it is answered by one pass over the collection's documents, each record read once and the pass ending at the limit. regex is the same: a test on each candidate, and alone one pass over the documents. ne, nin, exists(true) and not of keyed conditions are read from the keys alone (the field's present rows less the operands' rows), no record read.

Every field a plan reads (a filter's, a sort's) is keyed on the first run, one after another, so the ordered walk answers a sort whose field had no keys yet. A sort on _id ascending is the collection's own id order; while rows retain that order it reads no key (_id descending still reads the column route).

The case-insensitive equality and prefix have keys of their own:

Method Matches Note
ieq(f, s) a string equal to s ASCII letters A-Z compared without case; É and é differ; a value that is not a string never matches
iin(f, texts) a string equal to one of texts as ieq
istartsWith(f, s) a string starting with s as ieq

They are answered from a folded key of the field (the text with A-Z lowered, laid out as the text's own value key), built by the first query that uses one and kept by every write after, so the run is a key probe or a prefix range: explain() shows a keyed route. An empty text (ieq(f, ""), istartsWith(f, "")) has no key and is tested on the documents. icontains reads the trigram keys above.

Comparing fields and dates

const Prices = db.collection("f1prices");
Prices.insertMany([{ n: 1, cost: 5, price: 9 }, { n: 2, cost: 8, price: 8 }, { n: 3, cost: 9, price: 4 }, { n: 4, cost: 1 }]);
Prices.gtField("price", "cost").pluck("n");                       // [1]
Prices.lteField("price", "cost").pluck("n");                      // [2, 3]
Prices.eqField("price", "cost").pluck("n");                       // [2]
const Events = db.collection("f1events");
const at = (d, h) => Date.UTC(2024, 2, d, h);
Events.insertMany([{ n: 1, at: at(5, 1) }, { n: 2, at: at(5, 23) }, { n: 3, at: at(6, 2) }, { n: 4, at: Date.UTC(2024, 3, 1) }]);
Events.onDay("at", new Date(at(5, 12))).pluck("n");               // [1, 2]
Events.onDay("at", new Date(at(6, 12)), 180).pluck("n");          // [2, 3]
Events.inMonth("at", 2024, 3).pluck("n");                         // [1, 2, 3]
Events.inMonth("at", 2024, 4).pluck("n");                         // [4]
Events.lastDays("at", 3).count();                                 // 0
Method Matches Note
eqField(a, b), neField(a, b) field a equals, differs from field b both fields hold a value; a number equals a number, a string a string
gtField(a, b), gteField, ltField, lteField order of a against b two numbers or two strings; read from the record once
onDay(f, date, zone?) the day of date f holds milliseconds; zone is the offset from UTC in minutes, east positive (default UTC)
inMonth(f, year, month, zone?) the month (1 to 12)
lastDays(f, n, zone?) today and the n - 1 days before it, up to now now is taken at every run, so a kept chain always asks for the window ending at that moment
between(f, from, to) from <= f <= to takes Dates; see Filters

The date methods become one range (f >= start and f < next) on the number keys of f.

const Blurbs = db.collection("blurbs");
Blurbs.text("body");
Blurbs.insertMany([{ body: "the quick brown fox" }, { body: "a quick red hen" }]);
Blurbs.match("body", "quick fox").count();          // 1
Blurbs.match("body", "fox hen", "any").count();     // 2
Blurbs.match("body", "quick").count();              // 2

Sorting and paging

const Nums = db.collection("nums");
Nums.insertMany([1, 2, 3, 4, 5, 6, 7].map(n => ({ n, parity: n % 2 ? "odd" : "even" })));
Nums.sort("n", -1).limit(3).pluck("n");             // [7, 6, 5]
Nums.sort("n").skip(2).limit(2).pluck("n");         // [3, 4]
Nums.sort("parity").sort("n", -1).pluck("n");       // [6, 4, 2, 7, 5, 3, 1]
const page1 = Nums.sort("n").page({ size: 3 });
page1.rows.map(r => r.n);                           // [1, 2, 3]
const page2 = Nums.sort("n").page({ size: 3, after: page1.next });
page2.rows.map(r => r.n);                           // [4, 5, 6]
const page3 = Nums.sort("n").page({ size: 3, after: page2.next });
page3.rows.map(r => r.n);                           // [7]
page3.next;                                         // null
Nums.sort("n").page({ size: 3, total: true }).total; // 7
Nums.page({ size: 2 }).rows.length;                 // 2
Array.from(Nums.pages({ size: 3 })).map(rows => rows.length); // [3, 3, 1]
Call Meaning
sort(f, dir) dir: 1, "asc", false ascending (default); -1, "desc", true descending; again for more keys
limit(n), skip(n) integers from 0
page({ size, after, before, total }) { rows, next, prev, total? }; pass next as after; next is null on the last page; no skip with a page
pages({ size }) an iterator of row arrays

A document without the sorted field comes last either way.

sort(field, direction, { nulls: "first" }) puts those documents (the field null or missing) first instead, in id order in either direction, then the values in the sort's order, ties in id order; { nulls: "last" } is the default. skip and limit cut the whole order. It is the chain's one plain sort of the collection's own field (several sort keys, sort("a").sort("b", -1), are fine without it: the first key's order, each run of equal values ordered by the next, a document with no value last on every key, ties in id order; page refuses them): with several sort keys, a joined or relational chain, page, pages, last() or an aggregate (sum, avg, min, max, values, facets, groupBy) it is refused with an error, never run as nulls last. A filtered, limited run reads the rows with no value from the field's absent keys (see "Null and missing") and the values from the sort walk; where the keys cannot answer, the sorted column route gives the same order.

Sampling and walking

const Deck = db.collection("f1deck");
Deck.insertMany(Array.from({ length: 20 }, (_, i) => ({ i, suit: i % 2 ? "red" : "black" })));
Deck.eq("suit", "red").sample(3).length;                          // 3
Deck.eq("suit", "red").sample(100).length;                        // 10
Deck.eq("suit", "red").random().suit;                             // "red"
Deck.eq("suit", "none").random();                                 // null
const seen = [];
Deck.sort("i").each((card, index) => { seen.push(card.i); return seen.length < 4; }); // 4
seen;                                                             // [0, 1, 2, 3]
const pages = [];
Deck.eq("suit", "black").sort("i").chunk(4, (cards, page) => { pages.push(cards.length); }); // 10
pages;                                                            // [4, 4, 2]
Deck.eq("suit", "red").each(card => { Deck.update(card._id, { $set: { suit: "seen" } }); }); // 10
Deck.eq("suit", "seen").count();                                  // 10
Call Answers
sample(n) n distinct documents of the answer picked at random, in the answer's order (all of them when fewer match)
random() one document of the answer at random, null when none match
each(fn) fn(document, index) for each document in the answer's order, stopping when fn returns false; returns how many it was given
chunk(n, fn) fn(documents, page) for each page of n documents, stopping when fn returns false; returns the documents given

Only the ids of the answer are held; documents are made for the rows handed out. each and chunk take their ids first, so fn may write to the collection: a document removed meanwhile is skipped, one added meanwhile is not part of the walk. The chain must answer documents, or the fields select names (no join, with or groupBy): a select chain hands out its plain rows.

Scopes

const Users = db.collection("f1users");
Users.insertMany([{ n: 1, status: "active", age: 20 }, { n: 2, status: "active", age: 40 }, { n: 3, status: "banned", age: 50 }]);
Users.scope("active", c => c.eq("status", "active"));
Users.scope("adults", c => c.gte("age", 30));
Users.active().pluck("n");                                        // [1, 2]
Users.adults().active().pluck("n");                               // [2]
Users.gt("n", 1).active().pluck("n");                             // [2]
Users.or(q => q.active(), q => q.adults()).pluck("n");            // [1, 2, 3]
(() => { try { Users.scope("eq", c => c); } catch (e) { return e.name; } })();  // "INVALID_ARGUMENT"

collection.scope(name, fn) keeps fn for the collection; name then exists only on that collection and its chains, and starts a chain (Users.active()) and adds to any chain of the collection (chain.active(), inside and, or and not too). fn(chain) may add any conditions, a sort or a limit, and scopes compose in any order. A name that is already a built-in method is INVALID_ARGUMENT; a scope called on a collection that did not declare it is a TypeError (the name is not there). Declaring a name again replaces the scope; two collections may declare one name differently.

Explain

const Logs = db.collection("f1logs");
Logs.insertMany([{ n: 1, level: "warn", tags: ["a"] }, { n: 2, level: "info", tags: [] }]);
const plan = Logs.eq("level", "warn").containsAll("tags", ["a"]).explain();
["direct", "columns"].includes(plan.route);                       // true
plan.conditions.map(c => c.op);                                   // ["eq", "containsAll"]
plan.residual.some(c => c.op === "containsAll");                  // true
const loose = Logs.ne("level", "warn").explain();
loose.route;                                                      // "direct"
loose.driver;                                                     // { field: "level", kind: "keyedSets" }
Logs.sort("n", -1).limit(1).explain().sort;                       // [{ field: "n", direction: "desc" }]

chain.explain() runs the chain once and answers a plain object from what the route decided: route ("direct": value keys read from the documents; "columns": a columnar copy of the collection; "relational": the SQL engine), driver ({ field, kind } with kind "id", "postings", "keyedSets" or "sortWalk", or null), conditions (every top-level condition as { op, field }, groups and elemMatch listing their conditions, a field comparison its other), residual (the conditions tested on each candidate's record), aggregate ("keys" or "rows": where a grouped or folded answer on the direct route came from), sort, skip, limit, and declined (why the direct route did not answer, when it did not). driver.kind is also "counter" (a count read from the keys' counters, no row read), and driver.keys is "trigram" when a substring condition read the trigram keys. The names declined can take are listed in chain-reference.md.

How a chain runs

A chain only records conditions until a terminal runs it. The direct route then reads value keys: a condition the keys answer exactly (equality, in, a range, startsWith, null and absent values, the case-folded forms, a sort) costs no record read; the conditions they cannot prove (residual) are tested on each candidate's record, which is read once, and the same read supplies the sort value, the selected fields or the number a fold needs. A field's keys are built by the first query that filters, sorts or groups on it (in that call, from the stored documents) and kept by every write after, unless the field is declared. A chain whose conditions have no keyed leaf is answered by one pass over the records (route stays "direct", driver is null and every such condition is in residual); the columns route answers only where the direct route declines or is switched off, and a few shapes with no safe walk are refused instead. explain() shows which happened. The full model, and what makes a query cost more, is in chain-reference.md.

Selecting and plucking

const Nums = db.collection("nums2");
Nums.insertMany([{ n: 1, g: "a" }, { n: 2, g: "b" }, { n: 3, g: "a" }]);
Nums.sort("n").pluck("n");                          // [1, 2, 3]
Nums.sort("n").select("n").all().map(r => r.n);     // [1, 2, 3]
Object.keys(Nums.sort("n").select("n").first());    // ["n"]
Nums.select("g").distinct().all().length;           // 2
Nums.distinct("g");                                 // ["a", "b"]
Nums.eq("g", "a").ids().length;                     // 2

Answering JSON straight from a chain

json() and jsonFirst() write the matching rows (or the first one) straight into a JSON body: no JS row object, no JSON.stringify walk. Returning the result from a function, or wrapping it in new Response(body, init), answers those exact bytes with no further copy:

function (event) {
  return db.collection("orders").eq("status", "paid").sort("_id", -1).json();
}

json() holds [...] ([] when nothing matches); jsonFirst() holds one row, or null. The value is a JsonBody: it has .length (the byte count) and .toString() (the JSON text), but no fields of its own to read or walk in JS, and it can only be read once (a second read, e.g. returning it twice, throws JsonBody already used). A chain that answers another shape than one row per document (select, a join, with, groupBy, distinct, a compound) throws naming the reason instead of answering the wrong shape. See chain-reference.md.

Raw cursors

A collection's cursor (collection.cursor()) walks its documents in id order, without a chain: seekFirst/seekLast/seekGE/seekLE/seekLT/seekPrefix/seekPrefixLast position it, next/prev step it, key() answers the id it stands on and value() the document there. value(field) reads one field of that document without building the whole document, as pluck does; a nested path ("a.b") reads down as pluck's own paths do. A field or a path step the record does not hold answers undefined.

const Nums = db.collection("nums3");
Nums.insertMany([{ n: 1, g: "a" }, { n: 2, g: "b" }]);
const cur = Nums.cursor();
const seen = [];
for (let ok = cur.seekFirst(); ok; ok = cur.next()) seen.push(cur.value("n"));
cur.close();
seen.sort((a, b) => a - b);                          // [1, 2]

Aggregates

const Scores = db.collection("scores");
Scores.insertMany([
  { team: "red", pts: 10 }, { team: "red", pts: 20 },
  { team: "blue", pts: 5 }, { team: "blue", pts: 7 }, { team: "blue", pts: 9 },
]);
Scores.sum("pts");                                  // 51
Scores.eq("team", "red").avg("pts");                // 15
Scores.eq("team", "none").avg("pts");               // null
Scores.min("pts");                                  // 5
Scores.max("pts");                                  // 20
Scores.eq("team", "blue").count();                  // 3
Scores.group_by("team").sum("pts");                 // { red: 30, blue: 21 }
Scores.group_by("team").count();                    // { red: 2, blue: 3 }
Scores.groupBy("team").sum("pts", "total").avg("pts", "mean").sort("team").all(); // [{ team: "blue", total: 21, mean: 7 }, { team: "red", total: 30, mean: 15 }]
Scores.groupBy("team").count("n").max("pts", "best").having("n", ">=", 3).all(); // [{ team: "blue", n: 3, best: 9 }]
Scores.facets("team").team.values;                  // [{ value: "blue", count: 3 }, { value: "red", count: 2 }]
Scores.facets(["team"], { order: "value" }).team.values.map(v => v.value); // ["blue", "red"]
Scores.gt("pts", 0).top(2, "pts").pluck("pts");     // [20, 10]
Call Answers
sum(f), avg(f), min(f), max(f) a number; sum of nothing is 0, the others null
group_by(f).sum(g) (avg, min, max, count) one object, a property per group value
groupBy(...f).count(alias).sum(g, alias)...having(alias, op, v).all() one plain row per group (SQL rules); op is =, !=, >, >=, <, <=; every aggregate column is a number (an integer when the result has no fraction, otherwise a double)
values(f, opts), facets(fields, opts) { values: [{ value, count }], more } per field; opts: top, order ("value" or "count"), missing, after
top(n, weight) the n rows of greatest weight, after startsWith on a field declared with top(field, weight)

sum, avg, min and max with a join or groupBy in the chain need groupBy (error sum, avg, min and max of a relational chain need groupBy).

Grouping, buckets and top rows

bucket, histogram, countDistinct and topPerGroup are answered by the aggregate stage of the document route, and so is a groupBy(f).count() of the whole collection (one group key, counts only, no condition): the field's value keys are walked in order with one exact count per value, no record is read (explain().aggregate is "keys"). Every other groupBy, sum, avg, min and max keeps the SQL route. Where the stage reads records (explain().aggregate is "rows"), each is read once: nothing is capped, and a value the stage does not cover (an array, object or date group key) runs the SQL route. The numbers are exact: integers add in int64 and only a real or an overflow turns a result into a double. A grouped (or distinct) integer comes back as a stored int64 field does: a number inside the safe range, else a BigInt, so a key at or past 2^53 is exact and never a rounded double.

A collection can also keep a group's count and sums as its writes happen. index.create with kind: "aggregate" declares one ({collection, name, group?, metrics, where?}); index.list with kind=aggregate lists the collection's declarations and index.drop takes one away over HTTP; a script does the same with collection.aggregate(name, {group, metrics, where}), collection.aggregates() and collection.dropAggregate(name), which accept and refuse the same values. A collection file's aggregates array ({name, group?, metrics, where?} per entry) declares the same through substrio deploy (one index.create per entry, idempotent on its own); substrio push carries the array in the version but does not declare it. An absent or empty group is the whole collection as one group; group otherwise names fields (a field path, or {field, step} for a numeric bucket); metrics names "count" or {sum: field} per metric name. Declaring builds its groups now, over the documents already there, before it answers the declaration's state: "ready" once built, "building" while a large collection is still being counted, "none" for no such name. Redeclaring the same name with the same shape changes nothing; a different shape under the same name is refused, typed, naming what differs. Each group keeps its row count and, per declared metric, an exact int64 sum, a double beside it and the number of rows that held a number there. Every write updates its row's group as it happens, so nothing is recounted: an insert adds its row, a delete takes it off, an edit of a summed field moves the numbers, and an edit that moves the group key takes the row off its old group and adds it to its new one. A collection that declares no aggregate pays nothing for the feature.

Where one declared aggregate's group fields and metrics cover the question, a groupBy(f).count(), .sum(g) or .avg(g) over the whole collection is answered from those kept groups, with no document read (explain().aggregate is "maintained"), however many documents sit in each group. The answer is the one the routes above give, to the last digit: the count is the group's own rows, and a sum stays exact in int64 until a real or an overflow widens it, exactly as the fold over records does. While a declaration is still "building", a read it would otherwise answer falls back to the routes above instead.

Everything else keeps the routes above and answers the same numbers. A condition, a join, a window, a fill, an arrival page, a min, max or countDistinct, a declaration its own where scopes, a group path or bucket step no declaration has, a sum of a field no metric names, and a declaration that is not "ready" yet all fall back. So does a group field whose values are not cleanly of one kind: a boolean, an empty text, an array element, a non-text string, a very long text, or dates mixed with plain numbers. A declared aggregate does not keep min or max, a sum over array elements, a group across collections, or an ordered list of the groups (a declaration asking for ordered is refused by type).

select(fields).distinct() over fields of the chain's own collection (no join, group or compound) is answered by the same stage and is not a relational chain (explain().route is "direct", and "relational" only where the stage declines); a whole-row distinct(), a distinct over joined fields and a compound keep the SQL route.

const Sales = db.collection("sales");
Sales.insertMany([
  { region: "north", amt: 10, at: 0 }, { region: "north", amt: 30, at: 3600000 },
  { region: "south", amt: 20, at: 10800000 }, { region: "south", amt: 5, at: 10801000 },
  { region: "south", amt: 40, at: 7200000 },
]);
Sales.groupBy("region").count().sum("amt", "total").sort("region").all(); // [{ region: "north", count: 2, total: 40 }, { region: "south", count: 3, total: 65 }]
Sales.gt("amt", 5).groupBy("region").avg("amt", "mean").min("amt", "lo").sort("mean", -1).limit(1).all(); // [{ region: "south", mean: 30, lo: 20 }]
Sales.countDistinct("region");                      // 2
Sales.sum("amt");                                   // 105
Sales.bucket("at", "1h").count().sort("at").all();  // [{ at: 0, count: 1 }, { at: 3600000, count: 1 }, { at: 7200000, count: 1 }, { at: 10800000, count: 2 }]
Sales.bucket("at", "30m", { fill: true }).count().all().length; // 7
Sales.histogram("amt", 20).sort("amt").all();       // [{ amt: 0, count: 2 }, { amt: 20, count: 2 }, { amt: 40, count: 1 }]
Sales.topPerGroup("region", 2, "amt", "desc").map(r => r.amt); // [30, 10, 40, 20]
Call Answers
bucket(f, step, { fill }) a group key: the number (or date, in ms) cut into buckets of step (a number, or '1h', '1d', '1w', '30m', '15s', '250ms'); each row is { f: bucket start, ...aggregates }; rows with no number are one null bucket; fill: true also answers the empty buckets between the first and the last (never built: a page of any number of buckets costs the page; only a fill sorted by an aggregate builds its buckets, and is an error past 16777216 of them)
histogram(f, step, { fill }) bucket(f, step, { fill }).count(): rows { f, count }
countDistinct(f) how many different values f holds (null and missing do not count)
topPerGroup(g, n, f, dir) the first n documents of each group of g in f's order (1, -1, 'asc', 'desc'), groups in value order; each group is an ordered key walk stopped at n; documents with no g are in no group; a chain of conditions only

Groups from the stage come out in group key order unless sort names a group key or an aggregate alias; a boolean group key is true or false there (the SQL route answers 1 and 0).

Relations

const Authors = db.collection("authors");
const Books = db.collection("books");
const Reviews = db.collection("reviews");
const Profiles = db.collection("profiles");
Authors.hasMany("books", Books, "authorId");
Authors.hasOne("profile", Profiles, "authorId");
Books.belongsTo("author", Authors, "authorId");
Books.hasMany("reviews", Reviews, "bookId");

const ann = Authors.insert({ name: "Ann" })._id;
const bo = Authors.insert({ name: "Bo" })._id;
Authors.insert({ name: "Cy" });
const a1 = Books.insert({ authorId: ann, title: "A1", year: 2001 })._id;
Books.insert({ authorId: ann, title: "A2", year: 2010 });
const b1 = Books.insert({ authorId: bo, title: "B1", year: 2005 })._id;
Reviews.insert({ bookId: a1, stars: 5 });
Reviews.insert({ bookId: a1, stars: 3 });
Reviews.insert({ bookId: b1, stars: 4 });
Profiles.insert({ authorId: ann, bio: "writes" });

Authors.sort("name").with("books").all().map(a => a.books.length);        // [2, 1, 0]
Authors.eq("name", "Ann").with("books.reviews").first().books.map(b => b.reviews.length).sort(); // [0, 2]
Authors.eq("name", "Ann").with("books", q => q.sort("year", -1).limit(1)).first().books[0].title; // "A2"
Authors.sort("name").withCount("books").all().map(a => a.booksCount);     // [2, 1, 0]
Authors.whereHas("books", q => q.gt("year", 2005)).pluck("name");         // ["Ann"]
Authors.whereNone("books").pluck("name");                                 // ["Cy"]
Authors.with("books").first().books.length;                               // 2
Authors.withCount("books").first().booksCount;                             // 2
Books.eq("title", "B1").with("author").first().author.name;               // "Bo"
Authors.eq("name", "Ann").with("profile").first().profile.bio;            // "writes"
Authors.eq("name", "Cy").with("profile").first().profile;                 // null
Authors.eq("name", "Ann").with("books").first() instanceof Authors;       // true
Authors.eq("name", "Ann").with("books").first().books[0] instanceof Books; // true
Call Meaning
Owner.hasMany(name, Target, key) Target.key holds the owner's _id; read as an array
Owner.hasOne(name, Target, key) the same, read as one row or null
Target.belongsTo(name, Owner, key) Target.key holds the owner's _id; read as the owner row or null
.with(name, ...) each row carries its related rows; "a.b" nests; with(name, q => q...) filters, sorts and limits the related rows of each parent
.withCount(name) <name>Count, no rows read (hasMany)
.whereHas(name, q => ...), .whereNone(name, q => ...) keep the rows with, or without, a related row
.whereHas(name, op, n) keep the rows whose number of related rows compares to n: op is '>=', '>', '=', '<' or '<=', n a whole number from 0 (hasMany and hasOne)

whereHas(name, op, n) counts per row the run keeps, by one lookup of the owner's count (the same count withCount reads), so it costs the rows asked about and never builds the related collection's key set. A row with no related row has 0 (whereHas('books', '<', 1) is whereNone('books')). It is a condition of the chain itself, run by all, first, count and ids when the related field's keys are exact (limit and skip are fine); with a sort (take the rows and sort them), inside and, or and not, with a function, on a belongsTo or a belongsToMany, or in a chain shape that must read the plan another way (a join, groups, columns), it throws.

A declaration answers the collection and a later one of the same name replaces it. Rows from with and withCount are instances of their collection (save, set and inc work), and so is each related row; the relation is a read-only property of the row that save() never stores. Rows of select and join chains are plain.

Joins, subqueries and compounds

const Cust = db.collection("cust");
const Ord = db.collection("ord");
Cust.insertMany([{ name: "ann", tier: "gold" }, { name: "bo", tier: "basic" }]);
Ord.insertMany([{ customer: "ann", total: 5 }, { customer: "ann", total: 7 }, { customer: "zed", total: 1 }]);
Ord.join("cust", "customer", "name", "who").sort("total").all().map(o => o.who && o.who.tier); // [null, "gold", "gold"]
Ord.innerJoin("cust", "customer", "name", "c").count();                  // 2
Ord.leftJoin("cust", "customer", "name", "c").count();                   // 3
Ord.leftJoin("cust", "customer", "name", "c").eq("c.tier", "gold").count(); // 2
Ord.inQuery("customer", Cust.eq("tier", "gold"), "name").count();        // 2
Ord.notInQuery("customer", Cust.eq("tier", "gold"), "name").count();     // 1
Ord.select("customer").union(Cust.select("name")).sort("customer").all().map(r => r.customer); // ["ann", "bo", "zed"]

join(collection, localField, foreignField, as) adds the first match as as (null for none). innerJoin drops rows with no match, leftJoin keeps them. These, inQuery, notInQuery, existsQuery, union, unionAll, intersect and except run with SQL rules and answer plain rows. A joined field is named alias.field ("c.tier"). Compound arms pair columns by position; sort, skip and limit go on the receiver, not on an arm.

Kept chains

const Pts = db.collection("pts");
Pts.insertMany([{ n: 5 }, { n: 50 }, { n: 500 }]);
const big = Pts.gt("n", db.arg(0));
big.count(10);                                 // 2
big.count(100);                                // 1
big.all(100).map(r => r.n);                    // [500]

SQL over a collection

const Staff = db.collection("staff");
Staff.insertMany([{ name: "ann", age: 31 }, { name: "bo", age: 20 }, { name: "cy", age: 47 }]);
Staff.query("SELECT name, age WHERE age > ? ORDER BY age", 30); // [{ name: "ann", age: 31 }, { name: "cy", age: 47 }]
Staff.query("SELECT COUNT(*) AS n, MAX(age) AS oldest");        // [{ n: 3, oldest: 47 }]
Staff.prepare("SELECT name WHERE age >= ? ORDER BY age").all(31).length; // 2

query(sql, ...args) takes ? values as extra arguments (not an array). A SELECT without FROM reads the collection. SQL reads a collection and never writes it; see sql.md.

Writing

const Items = db.collection("items");
const { _id } = Items.insert({ sku: "a", qty: 1 });
Items.update(_id, { qty: 2 }).qty;                         // 2
Items.update(_id, { $inc: { qty: 1 } }).qty;               // 3
Items.upsert({ sku: "b" }, { sku: "b", qty: 5 }).qty;      // 5
Items.upsert({ sku: "b" }, { sku: "b", qty: 6 }).qty;      // 6
Items.replace(_id, { sku: "a", qty: 10 }).qty;             // 10
Items.delete(_id);                                         // true
Items.delete(_id);                                         // false
Items.update(_id, { qty: 1 });                             // null
Items.replace(_id, { sku: "a" });                          // null
const ids = Items.insertMany([{ sku: "c", qty: 1 }, { sku: "d", qty: 2 }]);
ids.length;                                                // 2
Items.upsertMany("sku", [{ sku: "c", qty: 9 }, { sku: "e", qty: 3 }]).length; // 2
Items.updateMany(ids, { $inc: { qty: 1 } });               // 2
Items.updateWhere({ sku: "c" }, { $set: { hot: true } });  // 1
Items.findOneAndUpdate({ sku: "d" }, { $inc: { qty: 1 } }).qty; // 4
Items.findOneAndUpdate({ sku: "none" }, { $inc: { qty: 1 } });  // null
Items.deleteMany(ids);                                     // 2
Items.deleteWhere({ sku: "e" });                           // 1
Items.count();                                             // 1
Items.put({ _id: "pa1", qty: 1 }).qty;                     // 1
Items.put({ _id: "pa1", qty: 2 }).qty;                     // 2
Items.get("pa1").qty;                                      // 2
Items.putMany([{ _id: "pa1", qty: 3 }, { _id: "pa2", qty: 4 }]).length; // 2
Items.upsert("_id", { _id: "pa1", qty: 5 }).qty;          // 5
Items.upsertMany("_id", [{ _id: "pa1", qty: 6 }, { _id: "pa1", qty: 7 }]).length; // 2
Items.get("pa1").qty;                                     // 7
Items.deleteMany(["pa1", "pa2"]);                          // 2
Call Answers
insert(doc), add(doc) { _id }; a doc that names _id is ID_RESERVED (use put)
create(doc) the stored row; a doc that names _id is ID_RESERVED (use put)
update(id, ops, { if }) the row after the update; null when if does not hold or no document has the id
set(id, fields) the row; sets plain fields
upsert(filter, doc) the row: updates the first match, else inserts
put(doc), upsert('_id', doc) the row: doc._id (a base62 id string or a positive integer) is the id the document is stored under; a stored document takes doc's fields, else it is inserted under that id, in one descent of the collection's tree
putMany(docs), upsertMany('_id', docs) ids in order; the same id twice gives one row, the last values win
replace(id, doc) the row, or null when no document has the id; the document becomes exactly doc (system fields stay)
insertMany(docs) ids in order; all or nothing; an element that is not an object is a TypeError naming its index
insertMany(docs, { onConflict: "ignore" }) the ids of the documents inserted: one a unique field refuses is skipped instead of thrown
upsertMany(key, docs) ids in order; key is a field name or an array of names
updateMany(ids, ops, { if }), updateWhere(filter, ops, { if }) documents written; updateMany(filter, ops, { if }) with a filter object is updateWhere
findOneAndUpdate(filter, ops, { if }) the row after the update, or null
findOneAndDelete(filter) the removed row as a plain object, or null when none matches
delete(id) true when it existed
deleteMany(ids), deleteWhere(filter) documents deleted; deleteMany(filter) with a filter object is deleteWhere
chain.update(ops), chain.delete() documents written or removed: the rows the chain answers
chain.increment(field, n), chain.decrement(field, n) documents written: update({ $inc }) by n or -n, 1 by default

An id no document holds is not counted by the Many calls.

The own-id methods (put, putMany, upsert('_id', doc) and upsertMany('_id', docs)) store a document under the id it holds. The id encoding is base62. Each document must hold its own _id; returned ids are strings. The parser also accepts positive integer BigInts and safe whole numbers as id inputs. Own-id upserts merge fields into a stored document; use replace to replace its fields. $setOnInsert applies only when the upsert inserts.

$inc, $mul, $min and $max need a number: a field that holds a string (or any value that is not a number; a null or absent field counts from 0 for $inc) throws TYPE_MISMATCH, a delta that is not a number a TypeError, and neither writes anything, in update, updateMany, updateWhere, upsert and findOneAndUpdate alike.

Writes that return rows

insertMany(docs, { returning: true }) answers the inserted rows (records, in order) instead of their ids (onConflict: 'ignore' answers the rows of the documents inserted). A chain's update(ops, { returning: true }), increment(field, n, { returning: true }) and decrement(field, n, { returning: true }) answer the rows written, in order (a row a guard or an error left out is not in the list), instead of the count. The options object is the last argument and takes no other key. A row is the live record, as every write answers (update, upsert): nothing is read when the write returns, and its fields are the stored document as written when the script asks.

findOrCreate(filter, doc) answers { row, created }: the first match as it is (never written), else doc inserted. updateOrCreate(filter, update) answers { row, created }: the first match takes the plain fields of update, else the filter's plain equality fields merged with update's are inserted. Both are the upsert's single find and single insert.

Chain writes and filter forms

const Shop = db.collection("orm1shop");
Shop.unique("sku");
Shop.insertMany([
  { sku: "a", shelf: 1, qty: 1, tags: ["new", "hot"], lines: [{ p: "x", n: 2 }] },
  { sku: "b", shelf: 2, qty: 5, tags: ["old"], lines: [] },
  { sku: "c", shelf: 1, qty: 9, tags: [] },
]);
Shop.insertMany([{ sku: "a" }, { sku: "d", shelf: 2, qty: 2, tags: ["hot"] }], { onConflict: "ignore" }).length; // 1
Shop.gte("qty", 5).update({ $set: { big: true } });           // 2
Shop.eq("big", true).increment("qty");                        // 2
Shop.eq("sku", "b").decrement("qty", 2);                      // 1
Shop.eq("sku", "b").value("qty");                             // 4
Shop.eq("sku", "zz").value("qty");                            // undefined
Shop.eq("sku", "zz").missing();                               // true
Shop.eq("sku", "a").firstOrFail().qty;                        // 1
try {
  Shop.eq("sku", "zz").firstOrFail();
} catch (e) {
  e.name;                                                     // "NotFoundError"
}
Shop.updateMany({ sku: { $in: ["a", "d"] } }, { $inc: { qty: 10 } }); // 2
Shop.find({ sku: { $regex: "^[ab]$" } }).length;              // 2
Shop.find({ sku: { $regex: "^B", $options: "i" } }).length;   // 1
Shop.find({ sku: { $not: { $in: ["a", "b"] } } }).length;     // 2
Shop.find({ $nor: [{ sku: "a" }, { qty: { $lt: 10 } }] }).length; // 2
Shop.count({ tags: { $all: ["new", "hot"] } });               // 1
Shop.count({ tags: { $size: 1 } });                           // 2
Shop.count({ lines: { $elemMatch: { p: "x", n: { $gte: 2 } } } }); // 1
Shop.find({}, { sort: { shelf: 1, qty: -1 }, select: ["sku"] }).map(r => r.sku); // ["a", "c", "d", "b"]
Object.keys(Shop.find({ sku: "a" }, { select: ["sku", "qty"] })[0]); // ["sku", "qty"]
Shop.findOneAndDelete({ sku: "c" }).qty;                      // 10
Shop.findOneAndDelete({ sku: "c" });                          // null
Shop.deleteMany({ qty: { $gte: 11 } });                       // 2
Shop.sort("qty").limit(1).delete();                           // 1
Shop.count();                                                 // 0

A chain's update, delete, increment and decrement write the documents the chain answers: its conditions are the filter, and its sort, skip and limit choose the rows as they do for ids(). They go through the writer updateWhere and deleteWhere use and answer the number of documents written. They belong to a chain, never to the bare collection (where update and delete take an id), and a chain with join, with, groupBy, distinct or a compound is refused.

A filter object takes, beside a value (an equality) and $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists and $startsWith: $not: { operators } (every document the operators do not select, one without the field included), $regex: "pattern" with $options: "i" (what regex answers; a pattern that does not compile is INVALID_QUERY_SYNTAX), $all: [values] (containsAll), $size: n (size) and $elemMatch: { field: value or operators } (elemMatch), and at the top level $and, $or and $nor over arrays of filters. They are the conditions the chain methods of the same name build, so both forms answer alike.

A member holding null selects the documents without a value ($eq, $ne, $in and $nin take null too), and a dotted name ("address.city") is a nested path with the chain's rules: the scalar operators only, no array traversed, and a filter of only leaves no key answers ($ne, $exists, $nin) is refused. A filter the chain has no plan for is refused saying why, and no other engine answers it (a value or object of a kind the place does not take is a TypeError, an unknown or empty name, list or operator object INVALID_ARGUMENT, a form no run gives UNSUPPORTED_OPERATION): undefined, a BigInt, a RegExp object (write { $regex, $options }), $options other than "" or "i", an empty $all, $elemMatch over operators, null as the operand of a range, any operator but $eq, $ne, $in and $nin on _id, $all, $size and $elemMatch on a nested path, and the text, vector, geo and script operators ($text, $search, $knn, $near, $whereJS); use the chain's match and top for text.

find with select answers plain rows holding the selected fields, as a select chain does, not records. findOneAndDelete answers the document as it was, a plain object with its _id: the stored document is gone, so it is not a record to save().

Update operators

const Carts = db.collection("carts");
const cart = Carts.insert({ n: 1, name: "a", tags: ["x"], lo: 5, hi: 5, f: 2, old: 1, items: [{ sku: "a", qty: 1 }, { sku: "b", qty: 2 }] })._id;
Carts.update(cart, { $set: { name: "b" } }).name;                  // "b"
Carts.update(cart, { $unset: { old: 1 } }).old;                    // undefined
Carts.update(cart, { $inc: { n: 2 } }).n;                          // 3
Carts.update(cart, { $min: { lo: 3 } }).lo;                        // 3
Carts.update(cart, { $max: { hi: 9 } }).hi;                        // 9
Carts.update(cart, { $mul: { f: 3 } }).f;                          // 6
Carts.update(cart, { $push: { tags: { $each: ["y", "z"], $position: 0, $slice: 3 } } }).tags; // ["y", "z", "x"]
Carts.update(cart, { $addToSet: { tags: "x" } }).tags;             // ["y", "z", "x"]
Carts.update(cart, { $pull: { tags: "z" } }).tags;                 // ["y", "x"]
Carts.update(cart, { $pullAll: { tags: ["y"] } }).tags;            // ["x"]
Carts.update(cart, { $pop: { tags: 1 } }).tags;                    // []
Carts.update(cart, { $rename: { name: "title" } }).title;          // "b"
Carts.update(cart, { $setWhere: { items: { where: { sku: "a" }, set: { qty: 9 } } } }).items[0].qty; // 9
Carts.update(cart, { $pull: { items: { sku: "b" } } }).items.length; // 1
typeof Carts.update(cart, { $currentDate: { seen: true } }).seen;  // "string"
Carts.upsert({ code: "k" }, { code: "k", $setOnInsert: { born: "first" } }).born; // "first"
Carts.upsert({ code: "k" }, { code: "k", $setOnInsert: { born: "second" } }).born; // "first"
Operator Does
$set, $unset set fields, remove fields
$inc, $mul add, multiply a number (RangeError past int64)
$min, $max keep the smaller, larger of stored and given
$push append; { $each, $position, $slice } for several, a place, a cap
$addToSet append what is missing
$pull, $pullAll remove elements equal to a value, matching a condition or fields ({ sku: "b" }), or in a list
$pop 1 last, -1 first
$rename { old: "new" }
$currentDate { f: true }, now as an ISO string
$setWhere { arr: { where: { sku: "a" }, set: { qty: 9 } } }, sets fields of matching elements
$setOnInsert upsert and upsertMany only, with the document's own fields

Plain fields and operators cannot mix in one call (INVALID_ARGUMENT).

A dotted key in $set, $unset, $inc, $mul, $min, $max (or in a plain field object, which is a $set) writes the member of a document: { $set: { "address.city": "Paris" } } sets member city of the document address, creating the documents of a missing step (geo.lat.deg makes geo, then lat, then deg). A step that is a scalar, null or an array throws and writes nothing; $unset of a path that is not there, and $min/$max that change nothing, write nothing. The member keeps its place in the document, the keys of the path move with the write, and a numeric operator on a missing member starts from 0 ($min, $max store the given value). A stored field whose literal name holds a dot (a whole document inserted with such a key) is read whole and wins over the path on read. A member's own name holding a dot cannot be addressed by a write, and the array operators, $rename and $setWhere do not take a dotted key (they throw). Only the written member is a new value (stored as the same operator stores it on a top-level field: a whole double is an integer): every other member of every document on the path is copied as its stored bytes and type, never decoded (an int64 past 2^53, -0, a double, a date, binary, a decimal, an array of mixed types and an empty document are unchanged).

$setWhere writes the same way: each matching element (it must be a document) is rebuilt with only the fields of set written, as the one number rule stores them; every other member of every element, and every element that did not match, is copied as its stored bytes and type (an int64 under 2^31, a double, a date, binary, a decimal and an object id stay exactly as they were). A dotted name in set writes a path, as a dotted $set does.

What a JS value stores. One rule, at the top level, nested in an object and inside an array, through insert, insertMany, put, upsert, update (plain or $set, dotted or not) and updateWhere, read back by get, a chain all() and select: an integer in int32 is an int32, a larger one an int64 (a BigInt past 2^53 exact); a fraction is a double; -0 stores 0 (JSON has no negative zero); NaN and the infinities store null; a Date stores its ISO text (a string, never a date type); a typed array stores as an object of its indexes and an ArrayBuffer as {}; the holes of a sparse array store null; undefined is no member of an object, null in an array and null when an update names it. An integer from 2^53 up stores as the integer of its JSON text on insert; a top-level update or $set of the same number stores the double.

Records and save

const Tasks = db.collection("tasks");
const task = Tasks.create({ title: "write", done: false, tries: 0 });
const mine = Tasks.get(task._id);
mine.done = true;
mine.dirty();                                  // true
mine.save();                                   // true
mine.dirty();                                  // false
Tasks.get(task._id).done;                      // true

const stale = Tasks.get(task._id);
Tasks.update(task._id, { $inc: { tries: 1 } });
stale.title = "rewrite";
try {
  stale.save();
} catch (e) {
  e.name;                                      // "ConflictError"
  e.code;                                      // "conflict"
}
stale.reload();
stale.title = "rewrite";
stale.save();                                  // true
Tasks.get(task._id).title;                     // "rewrite"
Tasks.get(task._id).tries;                     // 1

Every row is a record: assign fields, then save(). See records.md for every method.

Atomic and conditional writes

const Jobs = db.collection("jobs");
const job = Jobs.insert({ state: "ready", stock: 3 })._id;
Jobs.update(job, { state: "running" }, { if: { state: "ready" } }).state;  // "running"
Jobs.update(job, { state: "running" }, { if: { state: "ready" } });        // null
const v = Jobs.get(job)._version;
Jobs.update(job, { stock: 2 }, { if: { _version: v } }).stock;             // 2
Jobs.update(job, { stock: 1 }, { if: { _version: v } });                   // null
Jobs.update(job, { $inc: { stock: -1 } }, { if: { stock: { $gt: 0 } } }).stock; // 1

Jobs.insertMany([{ queue: "q", state: "ready", at: 2 }, { queue: "q", state: "ready", at: 1 }]);
Jobs.eq("queue", "q").eq("state", "ready").sort("at").claim(1, { $set: { state: "taken" } }).map(j => j.at); // [1]
Jobs.eq("queue", "q").eq("state", "ready").sort("at").claim(5, { $set: { state: "taken" } }).map(j => j.at); // [2]
Jobs.eq("queue", "q").eq("state", "ready").sort("at").claim(5, { $set: { state: "taken" } });                 // []

let runs = 0;
const charge = () => db.once("order-42", { keep: 3600000 }, () => { runs++; return { charged: 42 }; });
charge().replayed;                             // false
charge().replayed;                             // true
charge().reply.charged;                        // 42
runs;                                          // 1
  • update, updateMany, updateWhere, findOneAndUpdate take { if: cond }: a field value or { $eq, $ne, $gt, $gte, $lt, $lte, $exists }, all ANDed. A failed guard writes nothing.
  • { if: { _version: v } } is compare-and-set. A row's save() does it for you.
  • chain.claim(n, ops, ...args) takes up to n rows in the chain's order, applies ops, answers them after the update. Two claimers never get one row. Refused: join, select, groupBy, skip, limit, or, not, in, between, like.
  • db.once(key, { keep: ms }, fn) runs fn once per key and answers { reply, replayed }. fn must be synchronous and its reply JSON.

Indexes

const Events = db.collection("events");
Events.unique("ref");
Events.insertMany([{ ref: "a", at: 3 }, { ref: "b", at: 1 }]);
Events.get(Events.eq("ref", "b").first()._id).ref;   // "b"
Events.eq("ref", "b").count();                       // 1
Events.sort("at").pluck("ref");                      // ["b", "a"]
Events.gt("at", 2).pluck("ref");                     // ["a"]
try {
  Events.insert({ ref: "a" });
} catch (e) {
  e.name;                                            // "UniqueError"
  e.field;                                           // "ref"
}

_id is always a direct lookup. A field not declared is keyed by the first query that filters or sorts on it, which may pay to build its key. A field declared (index.create with kind: "collection", or an entry of indexes in the collection's file) is keyed before the declaration answers and kept keyed. GET /v1/_indexes?kind=collection&collection=c&field=f reports the state. Declarations that exist: unique(f) (the same as index.create with unique: true; unique(["a", "b"]) makes the pair unique, unique(["a", "b"], { nulls: "count" }) counts a null or missing member as a value, and the UniqueError then also has fields and values), text(f) (a field searched by match), top(f, weight) (a prefix top-k field), keyword(f, termBytes?) (a string field whose words are keyed by their suffixes, so contains and icontains of a word read terms and no record), expire(f, { after }) (a document is gone after milliseconds after the date its field holds), elements(f, members) (keys each element object of an array field, for elemMatch), schema(definition), and the relations (hasMany, hasOne, belongsTo, belongsToMany) and scope(name, fn) above.

The field of unique, keyword and index.create (and of an entry of a collection file's indexes) is a top-level name or a dotted path into nested objects: Users.unique("stripe.account_id") keeps two users from holding one Stripe account id, and Docs.keyword("profile.bio") makes contains / icontains on the path read terms. A unique path refuses a duplicate on insert, on a replace, and on a write that gives its parent object ({ stripe: { account_id: v } }) a held value; the UniqueError's field is the dotted path.

Errors

const Accounts = db.collection("accounts");
Accounts.unique("email");
Accounts.insert({ email: "a@x" });
try {
  Accounts.insert({ email: "a@x" });
} catch (e) {
  e.name;                                      // "UniqueError"
  e.field;                                     // "email"
  e.value;                                     // "a@x"
}
try {
  Accounts.eq("email");
} catch (e) {
  e.name;                                      // "TypeError"
}
Accounts.update("zzzzzzzz", { n: 1 });             // null
try {
  Accounts.update(Accounts.insert({ n: "text" })._id, { $inc: { n: 1 } });
} catch (e) {
  e.name;                                      // "TYPE_MISMATCH"
}
try {
  Accounts.insertMany([{ n: 1 }, 5]);
} catch (e) {
  e.name;                                      // "TypeError"
  e.message;                                   // "insertMany[1]: a document must be an object"
}
try {
  Accounts.update(Accounts.insert({ n: 1 })._id, { $inc: { n: 9223372036854775807n } });
} catch (e) {
  e.name;                                      // "RangeError"
}
const row = Accounts.get(Accounts.insert({ email: "b@x" })._id);
Accounts.update(row._id, { email: "c@x" });
row.email = "d@x";
try {
  row.save();
} catch (e) {
  e.name;                                      // "ConflictError"
}
try {
  db.transaction(tx => { Accounts.insert({ email: "e@x" }); throw new Error("stop"); });
} catch (e) {
  e.message;                                   // "stop"
}
What goes wrong What throws
a field that is not a string, a missing value, a value of a type the call does not take, a filter that is not an object TypeError, the message names the call
a number out of range (limit or skip below 0 or not whole), an unknown or empty name, an unknown $operator INVALID_ARGUMENT, the message names the call
a form the call does not run: a kv or graph write inside db.transaction, a write after a join or a group, an operator nested where it has no meaning UNSUPPORTED_OPERATION
a stored value of another type than the operation takes TYPE_MISMATCH
a write that would repeat a unique field UniqueError with code "unique", collection, field, value (a compound adds fields, values)
save() of a row another writer changed ConflictError with code "conflict", id, expected, actual
insertMany or upsertMany with an element that is not an object TypeError insertMany[i]: a document must be an object, nothing written
$inc, $mul, $min or $max on a field that holds a value that is no number TYPE_MISMATCH, the row is not changed
$inc, $mul, $min or $max with a delta that is no number TypeError, the row is not changed
$inc or $mul past int64 RangeError
db.once called again inside its own fn OnceInProgress
a failed if guard, an unknown get id, update or replace of an id no document holds, a findOneAndUpdate that matches nothing, an empty claim nothing thrown: null for each, [] for the claim
firstOrFail() of a chain that matches nothing NotFoundError