Connecting the Database

DATABASE=mongodb+srv://hitarth:<PASSWORD>@cluster0.kytu52c.[](mongodb.net/natours?)retryWrites=true&w=majority&appName=Cluster0
  • The natours in the above is the name of the database that we want to connect to. See the image below as well.
    Pasted image 20240223164704.png

However, if we are using a local database instead of an online one then we can use this instead :

  • DATABASE_LOCAL=mongodb://localhost:27017
  • Pasted image 20240223165159.png
  • We also have to specify the name of the database, so that URL will finally become :
  • DATABASE_LOCAL=mongodb://localhost:27017/natours
  • We have to keep the mongod running in the background if decide to use the local database.
    • Pasted image 20240223165352.png

Installing the MongoDB Drivers :

Pasted image 20240223165533.png

Configuring Server.js for Atlas / Local Database

Following things are required : 

const mongoose = require('mongoose');

const DB = process.env.DATABASE.replace(
  '<PASSWORD>',
  process.env.DATABASE_PASSWORD,
);

/* ----------------- For Atlas Database ----------------- */
// the connect method returns a promise so we can use "then" to handle the promise
mongoose
  .connect(DB, {
    useNewUrlParser: true,
    useCreateIndex: true,
    useFindAndModify: false,
  })
  .then((con) => {
    console.log(con.connections);
    console.log('DB connection successful');
/* ------------------------------------------------------ */
  });

/* ----------------- For Local Database ----------------- */
mongoose
  .connect(process.env.DATABASE_LOCAL, {
    useNewUrlParser: true,
    useCreateIndex: true,
    useFindAndModify: false,
  })
  .then((con) => {
    console.log(con.connections);
    console.log('DB connection successful');
  });
/* ------------------------------------------------------ */

Final Server.js file after configuration :

const mongoose = require('mongoose');
const dotenv = require('dotenv');
const app = require('./app');

dotenv.config({ path: './config.env' }); // this will read all the config from the file and save them into node js enviornment variables
// console.log(process.env); // logs all the env variables to the console
console.log(process.env.NODE_ENV);

const DB = process.env.DATABASE.replace(
  '<PASSWORD>',
  process.env.DATABASE_PASSWORD,
);

/* ----------------- For Atlas Database ----------------- */
// the connect method returns a promise so we can use "then" to handle the promise
mongoose
  .connect(DB, {
    useNewUrlParser: true,
    useCreateIndex: true,
    useFindAndModify: false,
  })
  .then((con) => {
    console.log(con.connections);
    console.log('DB connection successful');
  });
/* ------------------------------------------------------ */

/* ----------------- For Local Database ----------------- */
// mongoose
//   .connect(process.env.DATABASE_LOCAL, {
//     useNewUrlParser: true,
//     useCreateIndex: true,
//     useFindAndModify: false,
//   })
//   .then((con) => {
//     console.log(con.connections);
//     console.log('DB connection successful');
//   });
/* ------------------------------------------------------ */

/* ------------------ Start the server ------------------ */
// const port = 3000;
// listen for requests

const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`App is running on port ' ${port}`);
});

/* ------------------------------------------------------ */

What is Mongoose :

Pasted image 20240224175312.png

Creating a Simple Tour Model

Creating a schema : 

const tourSchema = new mongoose.Schema({
  name: {
    type: String,
    required: [true, 'Tour Must have a name'], // value and error message, required is a VALIDATOR
    unique: true,
  },
  rating: {
    type: Number,
    default: 4.5,
  },
  price: {
    type: Number,
    required: [true, 'A tour must have a price'],
  },
});

Creating a model :

  • To Note : An instance of a model is called a Document
const Tour = mongoose.model('Tour', tourSchema); // this will create a model named "Tour" from the schema
// it is a convention to start the model name with a capital letter
  • Mongo will automatically create a new collection named tours (notice that this is a plural name) from the keyword Tour above.

Creating Documents and Testing the Model

  • Remember that an instance of a model is called a document.
const testTour = new Tour({
  name: 'The Forest Hiker',
  rating: 4.7,
  price: 497,
});

testTour
  .save()
  .then((doc) => console.log(doc))
  .catch((err) => {
    console.log('ERROR 💥⚠️', err);
  }); // this will save the document to the database
// this save method returns a promise so we can use "then" to handle the promise

MVC Architecture :

  • Model : It is concerned about application's data and its business logic.
  • Controller : Its function is to handle application request, interact with models and send back responses to the client. All of this is called the application logic.
  • View : It is necessary if we have a graphical interface in our app or in other words if we are building a server side rendered website

Pasted image 20240224202520.png

Application vs Business Logic :

Pasted image 20240224212410.png

Refactoring for MVC

Pasted image 20240227191101.png

  • We put the tourSchema and Tour model in a separate folder (models) in a file named tourModel.js
  • Then we only export the Tour model.
  • Now, we'll need this Tour model in the tourController
  • Importing the module : Pasted image 20240227191731.png

Creating and Saving Documents using tourController.js

Creating and Saving Documents. Using Async Await for promises.

Inside the toursController : 

exports.getTour = (req, res) => {
  console.log(req.params);
  // const id = req.params.id * 1;
  // const tour = tours.find((el) => el.id === id);

  // res.status(200).json({
  //   status: 'success',
  //   data: {
  //     tour,
  //   },
  // });
};

// async below means that this is a special function which will keep running in the background while performing the code that is inside it while the rest of the code keeps running in the event loop. This async function will not block the event loop.
/* const data = await someFunctionPromise(); */ // → this await will stop the code at this point until the promise "someFunctionPromise" is resolved. Once the promise is returned then the code is allowed to run again and the data get's stored in the "data" variable.

exports.createTour = async (req, res) => {
  try {
    /* const newTours = new Tour({});
  newTours.save(); */
    // this save method returns a promise

    /* OR */

    const newTour = await Tour.create(req.body);

    res.status(201).json({
      status: 'success',
      data: {
        tour: newTour,
      },
    });
  } catch (err) {
    res.status(400).json({
      status: 'failed',
      message: err,
    });
  }
  // catch has access to the error object. Now, what do we put here ? Remember that an error can occur because of validation error (not defining the required fields), it is one of the errors that will get catched here.
  // If we tried to create a document without the defining the required fields then the promise "Tour.create(req.body)" will be rejected, then it will enter the catch block.
};

Pasted image 20240228203128.png

  • Notice that duration and difficulty are not added to the final document, this is because these two fields are not in the schema, therefore they are not put in the database.
  • Therefore, everything else that is the part of the schema is simply ignored.

Reading Documents :

get all tours :

  • Tours.find(); returns a promise, so we'll have to use async await here as well.
exports.getAllTours = async (req, res) => {
  try {
    const tours = await Tour.find();
    res.status(200).json({
      status: 'success',
      results: tours.length,
      data: {
        tours,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};
  • Pasted image 20240302192405.png

get tour by ID :

Pasted image 20240302193039.png

  • We can see the _id field here

Now this ID will come in the form as seen in the router below in tourRoutes.js

  • Pasted image 20240302193312.png
    • Check the /:id in the highlighted line in the image above.
  • in tourController.js we'll use this id param using req.params.id : 
exports.getTour = async (req, res) => {
  try {
    const tour = await Tour.findById(req.params.id);
    // Tour.findOne({ _id: req.params.id }); // this works the same as the above line. findById() is just a helper function.

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

Updating a tour :

  • We are updating the tour based on an ID.
exports.updateTour = async (req, res) => {
  try {
    // req.body contains the updated tour
    const tour = await Tour.findByIdAndUpdate(req.params.id, req.body, {
      new: true, // sends back the updated tour to back to the client
      runValidators: true, // runs the validators again
    });

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

Pasted image 20240302201757.png

Deleting tours by ID :

exports.deleteTour = async (req, res) => {
  try {
    await Tour.findByIdAndDelete(req.params.id);
    res.status(204).json({
      status: 'success',
      data: null,
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};
  • Nothing will be returned on successful deletion of the tour.

Using Script to load data from a json file on to a mongo server :

This will be an independent script.

node .\dev-data\data\import-dev-data.js --import returns the following when we do console.log(process.argv) :
Pasted image 20240304004709.png
we can see that an array is logged and the --import is at the index [2].

const mongoose = require('mongoose');
const dotenv = require('dotenv');
const fs = require('fs');
const Tour = require('../../models/tourModel');

dotenv.config({ path: './config.env' }); // we use . and not ./.. because we are in the root folder of the project and not in the dev-data folder

const DB = process.env.DATABASE.replace(
  '<PASSWORD>',
  process.env.DATABASE_PASSWORD,
);

mongoose
  .connect(DB, {
    useNewUrlParser: true,
    useCreateIndex: true,
    useFindAndModify: false,
  })
  .then((con) => {
    console.log(con.connections);
    console.log('DB "import" connection successful');
  });

// Read JSON File
const tours = JSON.parse(
  fs.readFileSync(`${__dirname}/tours-simple.json`, 'utf-8'), // we use __dirname to get the current directory of the file we are in
);

// Import data into DB
const importData = async () => {
  try {
    await Tour.create(tours); // .create method can also take in an array of objects to create multiple documents
    console.log('Data successfully loaded');
	process.exit();
  } catch (err) {
    console.log(err);
  }
};

// Delete all data from DB
const deleteData = async () => {
  try {
    await Tour.deleteMany();
    console.log('Data successfully deleted');
    process.exit();
  } catch (err) {
    console.log(err);
  }
};

// node .\dev-data\data\import-dev-data.js --import
if (process.argv[2] === '--import') {
  importData();
} else if (process.argv[2] === '--delete') {
  deleteData();
}

console.log(process.argv);

then we use node .\dev-data\data\import-dev-data.js --import or node .\dev-data\data\import-dev-data.js --delete to perform the required action.

Filtering : Using Query String :

semi code block at the end of this topic

This is made very easy because of express.
Pasted image 20240304012323.png

To exclude a query and prevent it from being processed:

Pasted image 20240304220710.png

Advanced Filtering :

>, >=, <, <= operators :

Remember the syntax that we use for filtering : 

{ difficulty: 'easy', duration: { $gte: 5 } }

The query string will be written like this : 

127.0.0.1:3000/api/v1/tours?duration[gte]=5&difficulty=easy // [gte] is the operator

On using the above URL and doing console.log(req.query); we get this :

  • Pasted image 20240304223115.png
  • this is almost identical to the filter object that we wrote manually. However the $ symbol before gte is missing.

To add the $ sign we do this : 

let queryStr = JSON.stringify(queryObj);
    queryStr = queryStr.replace(/\b(gte|gt|lte|lt)\b/g, (match) => `$${match}`); // → this will add a $ sign in front of the gte, gt, lte, lt // we are using 'b' to match the exact word and not the part of the word and 'g' to match all the occurences of the word

Sorting :

127.0.0.1:3000/api/v1/tours?sort=price

  • On console.log(req.query) Pasted image 20240305004627.png
// 3) Sorting
    if (req.query.sort) {
      query = query.sort(req.query.sort);
    }
  • To sort in descending order, we add - symbol before price : 127.0.0.1:3000/api/v1/tours?sort=-price

Now, what if two or more documents have the same price ? We can then specify other values to sort them with, like this :

  • 127.0.0.1:3000/api/v1/tours?sort=price,ratingsAverage
  • Pasted image 20240305015330.png
    // 3) Sorting
    if (req.query.sort) {
      const sortBy = req.query.sort.split(',').join(' ');
      console.log(sortBy);

      query = query.sort(sortBy);
    }

Field Limiting :

Only displaying the data that the user requires.
127.0.0.1:3000/api/v1/tours?fields=name,duration,price

  • Pasted image 20240306235720.png
  • mongoose requires fields names separated by spaces
// 4) Field Limiting
    if (req.query.fields) {
      // console.log(req.query); // { fields: 'name,duration,price' }
      // mongoose requires fields names separated by spaces
      const fields = req.query.fields.split(',').join(' ');
      // query = query.select('name duration price'); // this is called projecting
      query = query.select(fields);
    }

else {
      query = query.select('-__v'); // excluding the __v field // we add the - sign to exclude the "__v" field
    }

To hide a property in the schema itself :

Suppose we have fields containing sensitive information like passwords then we can prevent them from being displayed in the JSON API using select: false property in the schema Model (here tourModel).
Pasted image 20240307003201.png

Pagination:

127.0.0.1:3000/api/v1/tours?page=2&limit=10
  • This limit basically means the amount of results that we want per page.
  • skip : it the amount of documents that we want to skip before we start reading the documents.
/* ------------------------------------------------------ */
    // 5) Pagination
    const page = req.query.page * 1 || 1; // default value is 1, '*1' is used to convert the string to a number

    const limit = req.query.limit * 1 || 100; // default value is 100
    const skip = (page - 1) * limit;
    // skip: is the amount of documents that we want to skip before we start reading the documents
    // page=2&limit=10 → 1-10, page 1; 11-20, page 2; 21-30, page 3
    // query = query.skip(2).limit(2);

    query = query.skip(skip).limit(limit);

    if (req.query.page) {
      const numTours = await Tour.countDocuments(); // This returns the number of documents (it actually returns a promise)
      if (skip >= numTours) throw new Error('This page does not exist');
      // we can throw an error here because we are already in the "try" block, so, if an error is thrown we'll immediately reach the catch block where the error message will be processed.
    }
    /* ------------------------------------------------------ */

    // Execute Query
     const tours = await query;

Aliasing : and Prefilling the query String :

Another feature that we can add to our API is to provide an alias route to a request that might be very popular and gets requested a lot of times.

URL for the "5 best cheapest tours" : 127.0.0.1:3000/api/v1/tours?limit=5&sort=-ratingsAverage, price

  • Now, this is a pretty long URL, but we want to define a short and memorizable route for this.
  • The solution is to run a middleware before we run the router.route('/top-5-cheap').get(tourController.getAllTours); handler.
  • This middleware function is then going to manipulate the query object that's coming in.

(full tourRoutes.js) below : 

const express = require('express');
const fs = require('fs');
const tourController = require('../controllers/tourController'); // all of the functions that we have in the tourController.js file are now in the tourController object

// we can also use destructuring to import the functions like this :
// const { getAllTours, getTour, createTour, updateTour, deleteTour } = require('./../controllers/tourController'); and then we can use them directly in the router.route() method

const router = express.Router(); // we can use tourRouter as a middleware

// here val is the value of the parameter in question
// router.param('id', tourController.checkID);

router
  .route('/top-5-cheap')
  .get(tourController.aliasTopTours, tourController.getAllTours);

router
  .route('/') // here we don't have to specify the full path, because we already did that when we mounted the router
  .get(tourController.getAllTours)
  // .post(tourController.checkBody, tourController.createTour);
  .post(tourController.createTour);

router
  .route('/:id')
  .get(tourController.getTour)
  .patch(tourController.updateTour)
  .delete(tourController.deleteTour);

// exporting the router
module.exports = router;

In tourRoutes.js we'll do : 

router
  .route('/top-5-cheap')
  .get(tourController.aliasTopTours, tourController.getAllTours);
  • here the aliasTopTours is responsible for what data is going to be passed out to .getAllTours

As soon as we hit the /top-5-cheap route, the first middleware that's going to be run is the aliasTopTours.
Pasted image 20240309101721.png
So what this is going to do is that it will set the following properties of the query object to these values that we specified here. Basically prefilling parts of the query object before we then reach the getAllTours handler.
Therefore, as soon as we reach the getAllTours function, the query object is already prefilled even if the user didn't put any of these parameters in the query string.
Pasted image 20240309111405.png

exports.aliasTopTours = (req, res, next) => {
  req.query.limit = '5';
  req.query.sort = '-ratingsAverage, price';
  req.query.fields = 'name, price, ratingsAverage, summary, difficulty';
  next();
};

Pasted image 20240309111047.png

Code till now :

tourController.js

const fs = require('fs');
const Tour = require('../models/tourModel');

/* ------------------------------------------------------ */
// const tours = JSON.parse(
//   fs.readFileSync(`${__dirname}./../dev-data/data/tours-simple.json`),
// );

// exports.checkID = (req, res, next, val) => {
//   if (req.params.id * 1 > tours.length) {
//     console.log(`Tour id is : ${val}`);
//     // if we do not have this return here, then express would send this response back (res.status wala response), then it will hit the next() function and then it will send the response again, which will result in an error

//     // hence if the return statement is executed the next() function will not be executed

//     return res.status(404).json({
//       status: 'fail',
//       message: 'Invalid ID',
//     });
//   }
//   next();
// };
/* ------------------------------------------------------ */

// exports.checkBody = (req, res, next) => {
//   if (!req.body.name || !req.body.price) {
//     return res.status(400).json({
//       status: 'fail',
//       message: 'Missing name or price',
//     });
//   }

//   next();
// };

class  APIFeatures {
  constructor(query, queryString) {
    this.query = query;
    this.queryString = queryString;
  }

  filter() {
    
  }
}

exports.aliasTopTours = (req, res, next) => {
  req.query.limit = '5';
  req.query.sort = '-ratingsAverage, price';
  req.query.fields = 'name, price, ratingsAverage, summary, difficulty';
  next();
};

exports.getAllTours = async (req, res) => {
  try {
    console.log(req.query);

    /* ------------------------------------------------------ */
    // Method 1 of filtering the data
    /* const tours = await Tour.find({
      duration: 5,
      difficulty: 'easy',
    }); */

    // Method 1.1 of filtering the data : using the query string
    // const tours = await Tour.find(queryObj); // → this will return all the tours that match the query string

    // Method 2 of filtering the data :
    /* const tours = await Tour.find()
    .where('duration')
    .equals(5)
    .where('difficulty')
    .equals('easy'); */
    /* ------------------------------------------------------ */

    // BUILD QUERY
    // 1) Filtering
    const queryObj = { ...req.query };
    const excludedFields = ['page', 'sort', 'limit', 'fields'];
    excludedFields.forEach((el) => delete queryObj[el]);
    /* ------------------------------------------------------ */
    // 2) Advanced Filtering
    let queryStr = JSON.stringify(queryObj);
    queryStr = queryStr.replace(/\b(gte|gt|lte|lt)\b/g, (match) => `$${match}`); // → this will add a $ sign in front of the gte, gt, lte, lt // we are using 'b' to match the exact word and not the part of the word and 'g' to match all the occurrences of the word

    let query = Tour.find(JSON.parse(queryStr));
    /* ------------------------------------------------------ */
    // 3) Sorting
    if (req.query.sort) {
      const sortBy = req.query.sort.split(',').join(' ');
      console.log(sortBy);

      query = query.sort(sortBy);
    }
    // else block below is commented as it messes up pagination for some reason ("createdAt" same hai har document me)
    // else {
    //   query = query.sort('-createdAt');
    // }
    /* ------------------------------------------------------ */
    // 4) Field Limiting
    if (req.query.fields) {
      // console.log(req.query); // { fields: 'name,duration,price' }
      // mongoose requires fields names separated by spaces
      const fields = req.query.fields.split(',').join(' ');
      // query = query.select('name duration price'); // this is called projecting
      query = query.select(fields);
    } else {
      query = query.select('-__v'); // excluding the __v field // we add the - sign to exclude the "__v" field
    }

    /* ------------------------------------------------------ */
    // 5) Pagination
    const page = req.query.page * 1 || 1; // default value is 1, '*1' is used to convert the string to a number

    const limit = req.query.limit * 1 || 100; // default value is 100
    const skip = (page - 1) * limit;
    // skip: is the amount of documents that we want to skip before we start reading the documents
    // page=2&limit=10 → 1-10, page 1; 11-20, page 2; 21-30, page 3
    // query = query.skip(2).limit(2);

    query = query.skip(skip).limit(limit);

    if (req.query.page) {
      const numTours = await Tour.countDocuments(); // This returns the number of documents (it actually returns a promise)
      if (skip >= numTours) throw new Error('This page does not exist');
      // we can throw an error here because we are already in the "try" block, so, if an error is thrown we'll immediately reach the catch block where the error message will be processed.
    }
    /* ------------------------------------------------------ */

    // Execute Query
    const tours = await query;

    // const tours = await Tour.find();

    res.status(200).json({
      status: 'success',
      results: tours.length,
      data: {
        tours,
      },
    });
  } catch (err) {
    console.log(err);
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

exports.getTour = async (req, res) => {
  try {
    const tour = await Tour.findById(req.params.id);
    // Tour.findOne({ _id: req.params.id }); // this works the same as the above line. findById() is just a helper function.

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

// async below means that this is a special function which will keep running in the background while performing the code that is inside it while the rest of the code keeps running in the event loop. This async function will not block the event loop.
/* const data = await someFunctionPromise(); */ // → this await will stop the code at this point until the promise "someFunctionPromise" is resolved. Once the promise is returned then the code is allowed to run again and the data get's stored in the "data" variable.

exports.createTour = async (req, res) => {
  try {
    /* const newTours = new Tour({});
  newTours.save(); */
    // this save method returns a promise

    /* OR */

    const newTour = await Tour.create(req.body);

    res.status(201).json({
      status: 'success',
      data: {
        tour: newTour,
      },
    });
  } catch (err) {
    res.status(400).json({
      status: 'fail',
      message: 'Invalid data sent!',
      error: err,
    });
  }
  // catch has access to the error object. Now, what do we put here ? Remember that an error can occur because of validation error (not defining the required fields), it is one of the errors that will get catched here.
  // If we tried to create a document without the defining the required fields then the promise "Tour.create(req.body)" will be rejected, then it will enter the catch block.
};

exports.updateTour = async (req, res) => {
  try {
    // req.body contains the updated tour
    const tour = await Tour.findByIdAndUpdate(req.params.id, req.body, {
      new: true, // sends back the updated tour to back to the client
      runValidators: true, // runs the validators again
    });

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

exports.deleteTour = async (req, res) => {
  try {
    await Tour.findByIdAndDelete(req.params.id);
    res.status(204).json({
      status: 'success',
      data: null,
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

tourRoutes.js : 

const express = require('express');
const fs = require('fs');
const tourController = require('../controllers/tourController'); // all of the functions that we have in the tourController.js file are now in the tourController object

// we can also use destructuring to import the functions like this :
// const { getAllTours, getTour, createTour, updateTour, deleteTour } = require('./../controllers/tourController'); and then we can use them directly in the router.route() method

const router = express.Router(); // we can use tourRouter as a middleware

// here val is the value of the parameter in question
// router.param('id', tourController.checkID);

router
  .route('/top-5-cheap')
  .get(tourController.aliasTopTours, tourController.getAllTours);

router
  .route('/') // here we don't have to specify the full path, because we already did that when we mounted the router
  .get(tourController.getAllTours)
  // .post(tourController.checkBody, tourController.createTour);
  .post(tourController.createTour);

router
  .route('/:id')
  .get(tourController.getTour)
  .patch(tourController.updateTour)
  .delete(tourController.deleteTour);

// exporting the router
module.exports = router;

Refactoring the API features : Using Classes

utils/apiFeatuers.js : 

class APIFeatures {
  constructor(query, queryString) {
    console.log(`query777 : ${query}`);
    console.log(`queryString777 : ${queryString}`);

    this.query = query;
    this.queryString = queryString;
  }

  filter() {
    /* ------------------------------------------------------ */
    // BUILD QUERY
    // 1) Filtering
    const queryObj = { ...this.queryString };
    const excludedFields = ['page', 'sort', 'limit', 'fields'];
    excludedFields.forEach((el) => delete queryObj[el]);
    /* ------------------------------------------------------ */
    // 2) Advanced Filtering
    let queryStr = JSON.stringify(queryObj);
    queryStr = queryStr.replace(/\b(gte|gt|lte|lt)\b/g, (match) => `$${match}`);

    this.query = this.query.find(JSON.parse(queryStr));
    // let query = Tour.find(JSON.parse(queryStr));
    /* ------------------------------------------------------ */
    return this; // this will return the entire object so that we can chain other methods like 'sort' and 'limit' to it
  }

  sort() {
    /* ------------------------------------------------------ */
    // 3) Sorting
    if (this.queryString.sort) {
      const sortBy = this.queryString.sort.split(',').join(' ');
      // console.log(sortBy);

      this.query = this.query.sort(sortBy);
    }
    return this;
  }

  limitFields() {
    // 4) Field Limiting
    if (this.queryString.fields) {
      const fields = this.queryString.fields.split(',').join(' ');
      this.query = this.query.select(fields);
    } else {
      this.query = this.query.select('-__v');
    }

    return this;
  }

  pagination() {
    /* ------------------------------------------------------ */
    // 5) Pagination
    const page = this.queryString.page * 1 || 1;

    const limit = this.queryString.limit * 1 || 100;
    const skip = (page - 1) * limit;

    this.query = this.query.skip(skip).limit(limit);
    /* ------------------------------------------------------ */
    return this;
  }
}

module.exports = APIFeatures;

tourController.js

const fs = require('fs');
const Tour = require('../models/tourModel');
const APIFeatures = require('../utils/apiFeatures');

exports.aliasTopTours = (req, res, next) => {
  req.query.limit = '5';
  req.query.sort = '-ratingsAverage, price';
  req.query.fields = 'name, price, ratingsAverage, summary, difficulty';
  next();
};

exports.getAllTours = async (req, res) => {
  try {
    // console.log(req.query);

    // Execute Query
    const features = new APIFeatures(Tour.find(), req.query)
      .filter()
      .sort()
      .limitFields()
      .pagination(); // this 'features' will have access to all the methods that we have defined in the 'APIFeatures' class
    // All of this chaining works only because all of these methods are returning 'this'
    // On chaining these methods we keep adding stuff to the query here until the end, by the end we simply await the query so that it can come back with all the documents.
    const tours = await features.query;

    // const tours = await Tour.find();

    res.status(200).json({
      status: 'success',
      results: tours.length,
      data: {
        tours,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

exports.getTour = async (req, res) => {
  try {
    const tour = await Tour.findById(req.params.id);

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

exports.createTour = async (req, res) => {
  try {
    const newTour = await Tour.create(req.body);

    res.status(201).json({
      status: 'success',
      data: {
        tour: newTour,
      },
    });
  } catch (err) {
    res.status(400).json({
      status: 'fail',
      message: 'Invalid data sent!',
      error: err,
    });
  }
};

exports.updateTour = async (req, res) => {
  try {
    const tour = await Tour.findByIdAndUpdate(req.params.id, req.body, {
      new: true,
      runValidators: true,
    });

    res.status(200).json({
      status: 'success',
      data: {
        tour,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

exports.deleteTour = async (req, res) => {
  try {
    await Tour.findByIdAndDelete(req.params.id);
    res.status(204).json({
      status: 'success',
      data: null,
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};
  • Notice the following code piece in it : 
    const features = new APIFeatures(Tour.find(), req.query)
      .filter()
      .sort()
      .limitFields()
      .pagination(); // this 'features' will have access to all the methods that we have defined in the 'APIFeatures' class
    // All of this chaining works only because all of these methods are returning 'this'
    // On chaining these methods we keep adding stuff to the query here until the end, by the end we simply await the query so that it can come back with all the documents.
    const tours = await features.query;

Aggregation Pipeline Matching and Grouping

This is a feature of MongoDB. Mongoose gives us access to it so we can use it in the Mongoose Driver.
https://www.mongodb.com/docs/manual/aggregation/
https://www.mongodb.com/docs/v3.4/reference/operator/aggregation/

  • $group: allows us to group documents together, basically using accumulators, and an accumulator is for example calculating average. So if we have five tours, each of them has a rating we can then calculate the average rating using group.

When _id: null

tourController.js

exports.getTourStats = async (req, res) => {
  try {
    const stats = await Tour.aggregate([
      {
        $match: { ratingsAverage: { $gte: 4.5 } }, // match is used to filter out documents
      },
      {
        $group: {
          _id: null, // id is null because we want to calculate the stats on all the tours together in one big group
          numTours: { $sum: 1 }, // $sum is an accumulator operator, 1 is added to the accumulator for each document
          numRatings: { $sum: '$ratingsQuantity' },
          averageRating: { $avg: '$ratingsAverage' }, // $avg is an accumulator operator
          averagePrice: { $avg: '$price' },
          minPrice: { $min: '$price' },
          maxPrice: { $max: '$price' },
        },
      },
    ]);

    res.status(200).json({
      status: 'success',
      data: {
        stats,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

tourRoutes.js

router.route('/tour-stats').get(tourController.getTourStats);

postman :

Pasted image 20240314204325.png

When _id != NULL : Grouping Documents by a property + Multiple matching

exports.getTourStats = async (req, res) => {
  try {
    const stats = await Tour.aggregate([
      {
        $match: { ratingsAverage: { $gte: 4.5 } }, // match is used to filter out documents
      },
      {
        $group: {
          // _id: null, // id is null because we want to calculate the stats on all the tours together in one big group
          _id: { $toUpper: '$difficulty' }, // we can group the documents based on some fields, here we are grouping the documents based on the difficulty of the tours
          // "toUpper" is a built-in function in MongoDB to convert the difficulty to uppercase
          numTours: { $sum: 1 }, // $sum is an accumulator operator, 1 is added to the accumulator for each document
          numRatings: { $sum: '$ratingsQuantity' },
          avgRating: { $avg: '$ratingsAverage' }, // $avg is an accumulator operator
          avgPrice: { $avg: '$price' },
          minPrice: { $min: '$price' },
          maxPrice: { $max: '$price' },
        },
      },
      {
        $sort: { avgPrice: 1 }, // 1 for ascending order
      },
      {
        $match: { _id: { $ne: 'EASY' } }, // we can use match again to filter out the documents
      },
    ]);

    res.status(200).json({
      status: 'success',
      data: {
        stats,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};

Output : 

{
    "status": "success",
    "data": {
        "stats": [
            {
                "_id": "MEDIUM",
                "numTours": 3,
                "numRatings": 70,
                "avgRating": 4.8,
                "avgPrice": 1663.6666666666667,
                "minPrice": 497,
                "maxPrice": 2997
            },
            {
                "_id": "DIFFICULT",
                "numTours": 2,
                "numRatings": 41,
                "avgRating": 4.6,
                "avgPrice": 1997,
                "minPrice": 997,
                "maxPrice": 2997
            }
        ]
    }
}

getMonthlyPlan :

  • $unwind : what unwind is going to do is basically deconstruct an array field from the info documents and then output one document for each element of the array

    • basically we want to have one tour for each of these dates in the array

  • Pasted image 20240315201756.png

    • Basically we want a different tour for each of the startDates

  • Pasted image 20240315203236.png

exports.getMonthlyPlan = async (req, res) => {
  try {
    const year = req.params.year * 1; // 2021
    const plan = await Tour.aggregate([
      {
        $unwind: '$startDates',
      },
      {
        $match: {
          startDates: {
            $gte: new Date(`${year}-01-01`),
            $lte: new Date(`${year}-12-31`),
            // this will give us all the tours that have start dates between the first and the last day of the year
          },
        },
      },
      {
        $group: {
          _id: { $month: '$startDates' }, // grouping criteria
          numTourStarts: { $sum: 1 }, // info that we want to extract
          tours: { $push: '$name' }, // to get the names of all the tours that start in a particular month
        },
      },
      {
        $addFields: { month: '$_id' }, // to add a new field to the document
      },
      {
        $project: { _id: 0 }, // to hide the _id field (we yse 0 to hide the field and 1 to show the field)
      },
      {
        $sort: { numTourStarts: -1 }, // to sort the documents based on the number of tours that start in a particular month
      },
      {
        $limit: 12, // to limit the number of documents that are returned
      },
    ]);

    res.status(200).json({
      status: 'success',
      data: {
        plan,
      },
    });
  } catch (err) {
    res.status(404).json({
      status: 'fail',
      message: err,
    });
  }
};
  • Output :
    Pasted image 20240315205457.png

Virtual Properties :

  • They are basically fields that we can define in the schema, but they do not get persisted, so they will not be saved into the database in order to save us some space there.
  • Virtual Properties make a lot of sense for properties that can be derived from one another, for e.g. conversion from miles to kilometer. It doesn't make sense to store these two fields in the database if we can easily convert one to another.
  • We have to explicitly define in our schema that we want the virtual props to be included.
    Pasted image 20240315222615.png

Pasted image 20240315222746.png

// this virtual property will be created every time we "get" data from the database
tourSchema.virtual('durationWeeks').get(function () {
  return this.duration / 7; // this here refers to the current document
});

Full tourModel.js : 

const mongoose = require('mongoose');

const tourSchema = new mongoose.Schema(
  {
    name: {
      type: String,
      required: [true, 'Tour Must have a name'], // value and error message, required is a VALIDATOR
      unique: true,
      trim: true, // removes all the white spaces from the beginning and the end of the string
    },
    duration: {
      type: Number,
      required: [true, 'A tour must have a duration'],
    },
    maxGroupSize: {
      type: Number,
      required: [true, 'A tour must have a max group size'],
    },
    difficulty: {
      type: String,
      required: [true, 'A tour must have a difficulty'],
    },
    ratingsAverage: {
      type: Number,
      default: 4.5,
    },
    ratingsQuantity: {
      type: Number,
      default: 0,
    },
    price: {
      type: Number,
      required: [true, 'A tour must have a price'],
    },
    priceDiscount: Number,
    summary: {
      type: String,
      trim: true, // removes all the white spaces from the beginning and the end of the string
    },
    description: {
      type: String,
      trim: true,
    },
    imageCover: {
      type: String,
      required: [true, 'A tour must have a cover image'],
    },
    images: [String], // this is an array of strings
    createdAt: {
      type: Date,
      default: Date.now(),
      select: false, // this will not show the createdAt field in the response
    },
    startDates: [Date], // this is an array of dates // different dates when the tour starts // mongo will convert/parse the string into a date
    secretTour: {
      type: Boolean,
      default: false,
    },
  },
  {
    toJSON: { virtuals: true }, // this will include the virtual properties in the response
    toObject: { virtuals: true }, // toObject is used when the data is outputted as an object
  },
);

// this virtual property will be created every time we "get" data from the database
tourSchema.virtual('durationWeeks').get(function () {
  return this.duration / 7; // this here refers to the current document
});

const Tour = mongoose.model('Tour', tourSchema); // this will create a model named "Tour" from the schema
// it is a convention to start the model name with a capital letter

module.exports = Tour;

Note that we cannot use virtual properties in a query since they are not a part of the database

Middleware :

Just like express mongoose also has the concept of middleware.

1. Document Middleware

  • Document Middleware acts on the currently processed document.
  • Just like the virtual properties we define the middleware on the schema
  • pre middleware runs before an actual event
  • .pre() runs before .save() and .create(), and it does not get invoked before other commands like .insertMany().
  • post middleware has access to the document that was just saved to the database and the next middleware
  • post middleware functions are executed after all the pre middleware function
  • We can have multiple pre and post middlewares for the same hook.
  • hook for example is 'save' in the example code below.

Example :

in tourModel.js

tourSchema.pre('save', function () {
  console.log(this); // this will show the document that is being saved/processed.
});

On creating a new tour :
Pasted image 20240315230455.png
We'll get the following output in the console before the document is saved/created :
Pasted image 20240315230534.png

Creating a SLUG + POST middleware :

  • Slug is basically a string that we can put in the URL, usually based on some string like the name.
  • Installing slug : npm i slugify

tourModel.js : 

// .pre *runs before* `.save()` and `.create()`, and it does not get invoked before other commands like `.insertMany()`.
tourSchema.pre('save', function (next) {
  // console.log(this); // this will show the document that is being saved/processed.
  this.slug = slugify(this.name, { lower: true });
  next(); // this is a middleware, so we need to call next() to move to the next middleware
});

// we can have multiple middlewares for the same hook (pre or post)
tourSchema.pre('save', (next) => {
  console.log('Will save document...');
  next();
});

// post middleware has access to the document that was just saved to the database and the next middleware
tourSchema.post('save', (doc, next) => {
  console.log(doc);
  next();
});

Pasted image 20240316005941.png

Full Code (tourModel.js)

const mongoose = require('mongoose');
const slugify = require('slugify');

const tourSchema = new mongoose.Schema(
  {
    name: {
      type: String,
      required: [true, 'Tour Must have a name'], // value and error message, required is a VALIDATOR
      unique: true,
      trim: true, // removes all the white spaces from the beginning and the end of the string
    },
    slug: String,
    duration: {
      type: Number,
      required: [true, 'A tour must have a duration'],
    },
    maxGroupSize: {
      type: Number,
      required: [true, 'A tour must have a max group size'],
    },
    difficulty: {
      type: String,
      required: [true, 'A tour must have a difficulty'],
    },
    ratingsAverage: {
      type: Number,
      default: 4.5,
    },
    ratingsQuantity: {
      type: Number,
      default: 0,
    },
    price: {
      type: Number,
      required: [true, 'A tour must have a price'],
    },
    priceDiscount: Number,
    summary: {
      type: String,
      trim: true, // removes all the white spaces from the beginning and the end of the string
    },
    description: {
      type: String,
      trim: true,
    },
    imageCover: {
      type: String,
      required: [true, 'A tour must have a cover image'],
    },
    images: [String], // this is an array of strings
    createdAt: {
      type: Date,
      default: Date.now(),
      select: false, // this will not show the createdAt field in the response
    },
    startDates: [Date], // this is an array of dates // different dates when the tour starts // mongo will convert/parse the string into a date
    secretTour: {
      type: Boolean,
      default: false,
    },
  },
  {
    toJSON: { virtuals: true }, // this will include the virtual properties in the response
    toObject: { virtuals: true }, // toObject is used when the data is outputted as an object
  },
);

// this virtual property will be created every time we "get" data from the database
// we are not using arrow functions here because we want to use "this" keyword
tourSchema.virtual('durationWeeks').get(function () {
  return this.duration / 7; // this here refers to the current document
});

// .pre *runs before* `.save()` and `.create()`, and it does not get invoked before other commands like `.insertMany()`.
tourSchema.pre('save', function (next) {
  // console.log(this); // this will show the document that is being saved/processed.
  this.slug = slugify(this.name, { lower: true });
  next(); // this is a middleware, so we need to call next() to move to the next middleware
});

// we can have multiple middlewares for the same hook (pre or post)
tourSchema.pre('save', (next) => {
  console.log('Will save document...');
  next();
});

// post middleware has access to the document that was just saved to the database and the next middleware
tourSchema.post('save', (doc, next) => {
  console.log(doc);
  next();
});

const Tour = mongoose.model('Tour', tourSchema); // this will create a model named "Tour" from the schema
// it is a convention to start the model name with a capital letter

module.exports = Tour;

2. Query Middleware :

They allow us to run a function before or after a certain query is executed.

  • Let us suppose we some secret tours that are only accessible internally or to VIP people. Since these tours are secret we do not want them to ever appear in the result outputs.
  • So we will create a secret tour field and then query only for tours that are not secret using a middleware.
  • Pasted image 20240319014616.png
/* ----------------- QUERY MIDDLEWARE : ----------------- */

// tourSchema.pre('find', function (next) { // the problem with 'find' alone it that we'll still be able to access the secret tour by ID as 'findById()' behind the scenes is 'findOne()' only. One way to fix this is by duplicating the query middleware but using 'findOne()' instead. Another method is by using REGULAR EXPRESSIONS :
// /^find/ : this is a regular expression that will match all the strings that start with 'find' like 'findOne', 'findOneAndDelete', 'findOneAndUpdate', etc.
tourSchema.pre(/^find/, function (next) {
  this.find({ secretTour: { $ne: true } });

  this.start = Date.now(); // this will show the time when the query was executed
  next();
});

// since the post middleware has access to the documents that were returned from the database, we can use it to log the documents that were returned
tourSchema.post(/^find/, function (docs, next) {
  console.log(docs);
  console.log(`Query took ${Date.now() - this.start} milliseconds`);
  next();
});
/* ------------------------------------------------------ */
Interactive Graph
Table Of Contents
Connecting the Database
Installing the MongoDB Drivers :
Configuring Server.js for Atlas / Local Database
Final Server.js file after configuration :
What is Mongoose :
Creating a Simple Tour Model
Creating Documents and Testing the Model
MVC Architecture :
Application vs Business Logic :
Refactoring for MVC
Creating and Saving Documents using tourController.js
Reading Documents :
get all tours :
get tour by ID :
Updating a tour :
Deleting tours by ID :
Using Script to load data from a json file on to a mongo server :
Filtering : Using Query String :
To exclude a query and prevent it from being processed:
Advanced Filtering :
>, >=, <, <= operators :
Sorting :
Field Limiting :
To hide a property in the schema itself :
Pagination:
Aliasing : and Prefilling the query String :
Code till now :
Refactoring the API features : Using Classes
Aggregation Pipeline Matching and Grouping
When _id: null
When _id != NULL : Grouping Documents by a property + Multiple matching
getMonthlyPlan :
Virtual Properties :
Middleware :
1. Document Middleware
Example :
Creating a SLUG + POST middleware :
2. Query Middleware :