MongoDB CRUD, Querying, and Aggregation

Introduction

MongoDB is a document-oriented NoSQL database that stores records as BSON documents. BSON extends JSON with additional data types, including dates, ObjectIds, decimal values, and binary data. Unlike a relational table, a MongoDB collection does not require every document to have exactly the same fields. This flexibility is useful for rapidly changing applications, nested data, event records, catalogues, and content-management systems.

Flexibility does not eliminate the need for design. A professional MongoDB developer must choose suitable document boundaries, write precise queries, validate input, create appropriate indexes, and understand how updates and aggregation affect performance. This chapter develops those skills through the MongoDB shell syntax used by mongosh. Equivalent operations can be issued through official drivers such as the Node.js MongoDB driver.

Learning Objectives

By the end of this chapter, you should be able to:

• create, read, update, and delete MongoDB documents;
• use filters, projections, sorting, limiting, and pagination;
• query arrays, embedded documents, and nested fields;
• apply update operators safely and interpret update results;
• create and evaluate indexes;
• construct aggregation pipelines for transformation and analysis;
• distinguish operational queries from analytical aggregation;
• apply validation, security, and performance best practices.

1. Documents, Collections, and Data Modelling

A MongoDB document is a set of field-value pairs enclosed in braces. Documents have a unique _id field. If an application does not supply _id, MongoDB normally generates an ObjectId. An ObjectId contains a timestamp component and other identifying information; it should be treated as an identifier rather than as a guaranteed business timestamp.

{
  _id: ObjectId("65f100000000000000000001"),
  name: "Amina Rahman",
  email: "[email protected]",
  address: { city: "Nairobi", country: "Kenya" },
  skills: ["JavaScript", "MongoDB"],
  enrolledAt: ISODate("2025-01-15T00:00:00Z"),
  active: true
}

A collection is analogous to a table, but its documents may have different shapes. An embedded document, such as address, keeps related data together. An array, such as skills, represents a multi-valued attribute. Data should usually be embedded when it is bounded, read together, and owned by its parent. Referencing is preferable when related data is large, shared by many documents, updated independently, or unbounded in size.

MongoDB documents have a maximum BSON document size of 16 MiB. Consequently, embedding an unlimited number of comments, events, or log entries in one document is unsafe. A separate collection is more appropriate for unbounded relationships.

2. Create Operations

Select a database with use and insert documents into a collection. A collection is created implicitly on its first insert unless explicitly created earlier.

use training

db.students.insertOne({
  studentNo: "ST-104",
  name: "Amina Rahman",
  programme: "Full Stack Programming",
  scores: [78, 84, 91],
  contact: { city: "Nairobi", email: "[email protected]" },
  active: true
})

insertMany() inserts several documents:

db.students.insertMany([
  { studentNo: "ST-105", name: "David Otieno", programme: "Networking", active: true },
  { studentNo: "ST-106", name: "Lina Chen", programme: "Full Stack Programming", active: false }
])

The operation result reports inserted identifiers. In production, applications should validate required fields and enforce uniqueness where necessary. For example, a student number should normally have a unique index:

db.students.createIndex({ studentNo: 1 }, { unique: true })

The value 1 specifies ascending index order and -1 specifies descending order. A unique index rejects duplicate values, so it should be created only after existing data has been checked.

3. Read Operations: Filtering and Projection

The basic read operation is find(filter, projection). An empty filter matches every document; an empty projection returns all fields.

db.students.find({ programme: "Full Stack Programming" })

Equality can use either of these forms:

db.students.find({ active: true })
db.students.find({ active: { $eq: true } })

Common comparison operators include $gt, $gte, $lt, $lte, $ne, and $in.

db.students.find({ scores: { $gte: 80 } })
db.students.find({ programme: { $in: ["Full Stack Programming", "Data Analytics"] } })

Logical operators combine conditions:

db.students.find({
  $and: [
    { active: true },
    { scores: { $gte: 80 } }
  ]
})

db.students.find({
  $or: [
    { programme: "Networking" },
    { "contact.city": "Nairobi" }
  ]
})

Dot notation accesses nested fields. The query { "contact.city": "Nairobi" } matches a nested city field without retrieving the entire embedded object.

A projection controls returned fields and reduces network traffic. Include fields with 1 and exclude fields with 0:

db.students.find(
  { active: true },
  { _id: 0, studentNo: 1, name: 1, "contact.city": 1 }
)

Inclusion and exclusion cannot normally be mixed in one projection, except for excluding _id. Projection is not a security substitute for authorization: sensitive fields should also be protected through application access controls.

Sorting, limiting, and counting can be combined:

db.students.find({ active: true })
  .sort({ name: 1 })
  .skip(0)
  .limit(10)
  .count()

For stable pagination, sort by a deterministic key, often including _id. Offset pagination using large skip() values can become inefficient. Range-based pagination uses the last seen key, for example { _id: { $gt: lastId } }, together with a matching sort and index.

4. Arrays and Embedded Documents

MongoDB provides expressive array queries. A scalar condition may match when any array element satisfies it:

db.students.find({ scores: { $gte: 90 } })

$all requires all listed values to occur in the array:

db.students.find({ skills: { $all: ["JavaScript", "MongoDB"] } })

$size matches an exact array length:

db.students.find({ skills: { $size: 2 } })

When multiple conditions must apply to the same array element, use $elemMatch. Consider documents containing:

{ name: "Amina", assessments: [
  { module: "Databases", mark: 86 },
  { module: "Networks", mark: 62 }
] }

The following requires one assessment to have both properties:

db.students.find({
  assessments: { $elemMatch: { module: "Databases", mark: { $gte: 80 } } }
})

Without $elemMatch, separate conditions on array fields can potentially be satisfied by different array elements. This distinction is essential when querying arrays of embedded documents.

Nested fields can be filtered and projected together:

db.students.find(
  { "contact.city": "Nairobi" },
  { _id: 0, name: 1, "contact.email": 1 }
)

The $exists operator tests whether a field exists, while $type tests its BSON type. These operators are useful during data-quality audits, but applications should not rely on inconsistent types indefinitely.

5. Update Operations

updateOne() modifies the first matching document; updateMany() modifies all matching documents; replaceOne() replaces an entire document except for its immutable _id. Prefer update operators when only selected fields should change.

db.students.updateOne(
  { studentNo: "ST-104" },
  { $set: { active: false, "contact.city": "Mombasa" } }
)

Important update operators include:

For example:

db.students.updateOne(
  { studentNo: "ST-104" },
  {
    $addToSet: { skills: "Docker" },
    $inc: { loginCount: 1 },
    $set: { updatedAt: new Date() }
  }
)

An upsert updates a matching document or inserts one when no match exists:

db.courseStats.updateOne(
  { courseCode: "DB301", period: "2025-T1" },
  { $inc: { enrolments: 1 } },
  { upsert: true }
)

Upserts must be designed carefully. A poorly chosen filter can create duplicate logical records, especially when no unique index protects the key.

Update results typically include matchedCount, modifiedCount, and, for an upsert, upsertedId. A matched document may have modifiedCount: 0 when the requested value is already present.

6. Delete Operations

deleteOne() removes one matching document and deleteMany() removes all matching documents.

db.students.deleteOne({ studentNo: "ST-106" })
db.students.deleteMany({ active: false })

Deletion is irreversible unless backups, audit records, or a soft-delete policy exist. A soft delete uses a field such as deletedAt and excludes deleted records from normal queries. This supports recovery and auditability, but requires consistent filters and appropriate indexes.

Never execute an unreviewed deleteMany({}) in a production environment. Operational safeguards include role separation, confirmation workflows, backups, transactions where appropriate, and testing against a staging database.

7. Indexes and Query Performance

An index is an auxiliary data structure that helps MongoDB locate documents without scanning every document. Indexes improve reads but consume memory and storage and impose write-maintenance cost.

db.students.createIndex({ programme: 1, active: 1 })
db.students.getIndexes()
db.students.dropIndex({ programme: 1, active: 1 })

Compound index field order matters. A compound index on { programme: 1, active: 1 } supports queries beginning with programme and can support both fields efficiently. It is generally less useful for a query filtering only on active, because active is not the prefix.

Use explain() to inspect a query plan:

db.students.find({ programme: "Full Stack Programming", active: true })
  .explain("executionStats")

Useful indicators include totalDocsExamined, totalKeysExamined, and executionTimeMillis. An IXSCAN indicates index scanning; a COLLSCAN indicates a collection scan. A collection scan is not automatically wrong for a small collection, but it may become a bottleneck as data grows.

A covering query can be answered from an index alone when all required filter and projected fields are present in the index. Index design should be based on real query patterns, selectivity, sort requirements, write frequency, and memory limits—not on indexing every field.

8. Aggregation Pipelines

A normal find() retrieves matching documents. An aggregation pipeline transforms documents through ordered stages. Each stage receives the previous stage’s output and produces a new stream.

db.orders.aggregate([
  { $match: { status: "PAID" } },
  { $group: {
      _id: "$customerId",
      totalSpent: { $sum: "$amount" },
      orderCount: { $sum: 1 }
  } },
  { $sort: { totalSpent: -1 } },
  { $limit: 10 }
])

The principal stages are:

• $match: filters documents;
• $project: selects or computes fields;
• $set or $addFields: adds computed fields;
• $unset: removes fields;
• $unwind: creates one output document per array element;
• $group: groups documents and calculates accumulations;
• $sort: orders documents;
• $limit and $skip: restrict output;
• $lookup: performs a left outer join with another collection;
• $facet: runs multiple pipelines over the same input;
• $count: counts pipeline results;
• $out or $merge: writes results to a collection.

A $group stage uses _id as its grouping key. Accumulators include $sum, $avg, $min, $max, $first, $last, $push, and $addToSet. For a group containing values x₁, x₂, ..., x_n, the arithmetic mean is:

Here, x̄ is the mean, each xᵢ is a numeric observation, and n is the number of observations. The formula assumes numeric values and treats each observation with equal weight. MongoDB implements this calculation with $avg.

db.orders.aggregate([
  { $group: {
      _id: "$customerId",
      averageOrder: { $avg: "$amount" },
      largestOrder: { $max: "$amount" }
  } }
])

$unwind and Array Analysis

Suppose each order contains items: [{ sku, quantity, price }]. To calculate product revenue, unwind the array and compute line revenue:

db.orders.aggregate([
  { $unwind: "$items" },
  { $set: {
      lineRevenue: { $multiply: ["$items.quantity", "$items.price"] }
  } },
  { $group: {
      _id: "$items.sku",
      unitsSold: { $sum: "$items.quantity" },
      revenue: { $sum: "$lineRevenue" }
  } },
  { $sort: { revenue: -1 } }
])

$unwind changes one order with three items into three pipeline documents. This is powerful, but it can multiply the number of records and therefore affect memory and execution time.

Conditional and Date Transformations

Expressions such as $cond, $ifNull, $toDate, $year, and $month transform values. For example, monthly paid revenue can be calculated as follows:

db.orders.aggregate([
  { $match: { status: "PAID" } },
  { $group: {
      _id: {
        year: { $year: "$paidAt" },
        month: { $month: "$paidAt" }
      },
      revenue: { $sum: "$amount" },
      orders: { $sum: 1 }
  } },
  { $sort: { "_id.year": 1, "_id.month": 1 } }
])

Date calculations depend on timezone interpretation. Systems serving multiple regions should explicitly define the reporting timezone and store timestamps consistently, commonly in UTC.

$lookup

$lookup combines documents from another collection. For example, an order can be joined to a customer:

db.orders.aggregate([
  { $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customer"
  } },
  { $unwind: "$customer" },
  { $project: { orderNo: 1, amount: 1, "customer.name": 1 } }
])

Joins are useful, but frequent large joins may indicate that document boundaries or indexes need reconsideration. The foreign field should generally be indexed, and the pipeline should reduce unnecessary documents before the lookup.

9. Applied Case: Training-Commerce Dashboard

Background

This illustrative case concerns an online training provider storing orders in an orders collection. Each order contains customerId, status, paidAt, amount, and an items array. Management wants to identify its highest-value products and monitor monthly revenue.

Problem

A report based on application-side loops transfers many documents to the application server, duplicates business logic, and produces slow results during peak periods. The provider also needs reliable student and order identifiers.

Analysis and Solution

The team first creates unique indexes for business identifiers and a compound index supporting common filtering:

db.customers.createIndex({ email: 1 }, { unique: true })
db.orders.createIndex({ status: 1, paidAt: 1 })
db.orders.createIndex({ "items.sku": 1 })

The revenue report is implemented as a pipeline. $match restricts processing to paid orders, $unwind exposes each purchased item, $multiply calculates line revenue, $group aggregates by product, and $sort ranks products. The database therefore performs filtering and aggregation close to the data.

The team uses explain("executionStats") on representative workloads, checks that indexes are used, and tests reports with realistic order volumes. Data validation ensures that amount, quantity, and price are numeric and that paidAt is a date.

Outcome

The report returns product-level revenue and unit totals without transferring all raw orders to the application. Indexes improve selective filtering, while the aggregation pipeline provides a repeatable analytical definition. The team documents timezone rules, currency assumptions, and the meaning of a “paid” order so that managers interpret results consistently.

Learning Points

1. Model documents around access patterns and ownership.
2. Put selective $match stages early when possible.
3. Use $unwind deliberately because it expands data volume.
4. Treat indexes as performance tools with write and storage costs.
5. Define metrics, currencies, dates, and statuses explicitly.

10. Governance, Security, and Best Practices

MongoDB applications must prevent query injection. Do not construct query objects by blindly accepting arbitrary client JSON. Whitelist permitted fields and operators, validate types, and apply authorization before executing a query.

Use least-privilege database accounts, encrypted connections, strong credential management, network restrictions, auditing where required, and protected backups. Personal data should be collected only for legitimate purposes, retained for an appropriate period, and deleted or anonymized according to applicable law and organizational policy.

Schema flexibility should be controlled with validation. JSON Schema validation can require fields and types, for example:

db.createCollection("students", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["studentNo", "name", "active"],
      properties: {
        studentNo: { bsonType: "string" },
        name: { bsonType: "string" },
        active: { bsonType: "bool" }
      }
    }
  }
})

Use transactions when several changes must satisfy one atomic business rule. Transactions provide stronger consistency but add coordination and performance overhead; they should not replace good document modelling. For routine independent document updates, single-document atomicity is often sufficient.

Aggregation pipelines should be monitored for memory usage, cardinality growth, sort cost, and regularly changing query patterns. Test with production-like data, not merely a small development dataset. Keep database drivers and server versions supported, and review release documentation before relying on new operators.

Chapter Summary

MongoDB stores flexible BSON documents in collections, but effective systems still require deliberate modelling and validation. CRUD operations are expressed through insert, find, update, replace, and delete methods. Filters use comparison and logical operators; dot notation addresses nested fields; $elemMatch ensures multiple conditions apply to one array element; and projections reduce returned data.

Update operators such as $set, $inc, $addToSet, and $pull modify selected portions of documents. Unique and compound indexes support integrity and performance, while explain() reveals query-plan behaviour. Aggregation pipelines transform data through stages such as $match, $unwind, $group, $lookup, and $sort, enabling database-side reporting and analytical transformations. Professional use additionally requires authorization, validation, safe deletion, timezone and metric definitions, monitoring, and responsible handling of personal data.

Key Terms

Document: A BSON record containing field-value pairs.

Collection: A group of MongoDB documents.

BSON: Binary JSON format supporting additional data types.

ObjectId: A commonly used MongoDB identifier type.

CRUD: Create, read, update, and delete operations.

Projection: Selection of fields returned by a query.

Dot notation: Syntax for addressing nested fields.

$elemMatch: Operator requiring conditions to match one array element.

Upsert: Update-or-insert operation.

Atomicity: The property that an operation completes as one indivisible unit.

Index: Data structure that accelerates data access.

Compound index: Index containing multiple ordered fields.

Query plan: Strategy selected by MongoDB to execute a query.

Aggregation pipeline: Ordered stages that transform document streams.

Accumulator: Aggregation expression that combines values, such as $sum or $avg.

$unwind: Stage that expands array elements into separate documents.

$lookup: Aggregation stage that combines data from another collection.

Schema validation: Rules restricting document structure and data types.

Soft delete: Marking a record deleted instead of physically removing it.

Review Questions

Short Answer Questions

1. What is the difference between a MongoDB document and a collection?
2. Why is $elemMatch useful when querying arrays of embedded documents?
3. What is the purpose of a projection?
4. Distinguish updateOne() from updateMany().
5. What information can explain("executionStats") provide?

Long Answer Questions

1. Explain how document embedding and referencing support different access patterns. Include limitations of each approach.
2. Describe the major MongoDB update operators and give an appropriate use case for each.
3. Explain compound indexes, including why field order matters.
4. Describe the stages and reasoning of an aggregation pipeline that calculates product revenue from order items.
5. Discuss security, validation, and governance practices required for a production MongoDB application.

Analytical / Application Questions

1. Design a query and projection that returns active students in Nairobi whose skills include MongoDB, showing only their names and email addresses. Explain your choices.
2. An orders pipeline performs $unwind before filtering by status and date. Analyse the likely performance implications and propose a revised stage order.
3. A company stores an unlimited number of events inside each customer document. Evaluate this design against MongoDB document-size and update considerations, then recommend an alternative.

Multiple Choice Questions with Answers

Question 1

Which MongoDB method inserts several documents in one operation?

A. insertMany()

B. insertBatch()

C. createMany()

D. addDocuments()

Correct Answer: A. insertMany()

Question 2

Which projection is valid for returning only name and email while excluding the default identifier?

A. { name: 1, email: 1, _id: 0 }

B. { name: 0, email: 0, _id: 1 }

C. { name: 1, email: 0, _id: 0 }

D. { name: true, email: false }

Correct Answer: A. { name: 1, email: 1, _id: 0 }

Question 3

Which operator ensures that multiple conditions apply to the same element of an array of embedded documents?

A. $all

B. $elemMatch

C. $arrayMatch

D. $sameElement

Correct Answer: B. $elemMatch

Question 4

What does $addToSet do?

A. Adds an array element only if it is not already present

B. Adds an element at the beginning regardless of duplicates

C. Sorts every array after insertion

D. Removes all duplicate documents in a collection

Correct Answer: A. Adds an array element only if it is not already present

Question 5

What is the principal effect of a unique index?

A. It guarantees descending sort order

B. It prevents duplicate indexed values

C. It encrypts indexed fields

D. It automatically deletes old records

Correct Answer: B. It prevents duplicate indexed values

Question 6

Which aggregation stage creates one pipeline document for each element of an array?

A. $group

B. $project

C. $unwind

D. $facet

Correct Answer: C. $unwind

Question 7

In an aggregation pipeline, what is the usual purpose of an early $match stage?

A. To reduce the number of documents processed by later stages

B. To convert every document into an array

C. To create a unique index

D. To replace all missing fields

Correct Answer: A. To reduce the number of documents processed by later stages

Question 8

Which accumulator calculates the arithmetic mean of numeric values in a group?

A. $total

B. $meanValue

C. $average

D. $avg

Correct Answer: D. $avg

Question 9

What does an upsert do when no document matches its filter?

A. It always reports an error

B. It inserts a document based on the operation

C. It deletes the nearest document

D. It creates an index automatically

Correct Answer: B. It inserts a document based on the operation

Question 10

Which command is most appropriate for examining whether a query uses an index?

A. find().inspectPlan()

B. find().explain("executionStats")

C. find().showIndex()

D. find().profileQuery()

Correct Answer: B. find().explain("executionStats")