Diving deep into node js
Episode 05 - Diving into the NodeJS Github Repo
Hey everyone! Welcome back to the Node.js tutorial series. In this episode, we'll explore how modules actually work behind the scenes!
We'll dive into how modules load into a page and how Node.js handles multiple modules, focusing on a deep dive into the module.exports and require functions!
What we will cover:
- Behind the Scenes of Modules
- Function Scope in JavaScript
- IIFE (Immediately Invoked Function Expression)
- How require() Works Behind the Scenes
- Module Caching
- Exploring NodeJS GitHub Repository
- V8 Engine Integration
- Superpowers - libuv
- Built-in Modules in lib Directory
Behind the Scenes
In JavaScript, when you create a function...
function x() {
const a = 10;
function b() {
console.log("b");
}
}
Q: Will you be able to access this value outside the function?
console.log(a);
OUTPUT: ReferenceError: a is not defined
NO! You cannot access a value outside the function x because it is defined within the function's scope.
Each function creates its own scope, so variables inside a function are not accessible from outside that function.
Important Concept 🧐
Modules in Node.js work similarly to function scopes!
When you require a file, Node.js wraps the code from that file inside a function. This means that all variables and functions in the module are contained within that function's scope and cannot be accessed from outside the module unless explicitly exported.
To expose variables or functions to other modules, you use module.exports. This allows you to export specific elements from the module, making them accessible when required elsewhere in your application.
IIFE - Immediately Invoked Function Expression
All the code of a module is wrapped inside a function when you call require. This function is not a regular function; it's a special type known as an IIFE (Immediately Invoked Function Expression)!
Here's how it works:
(function() {
// All the code of the module runs inside here
})();
In this pattern, you create a function and then immediately invoke it!
This is different from a normal function in JavaScript, which is defined and then called separately:
// Normal Function (Define, then Call)
function x() {
console.log("Hello");
}
x(); // Called separately
// IIFE (Define AND Call at the same time)
(function() {
console.log("Hello");
})(); // Called immediately!
Why IIFE in Node.js?
In Node.js, before passing the code to the V8 engine, it wraps the module code inside an IIFE.
The purpose of IIFE is to:
- Immediately Invoke Code: The function runs as soon as it is defined
- Keep Variables and Functions Private: By encapsulating the code within the IIFE, it prevents variables and functions from interfering with other parts of the code. This ensures that the code within the IIFE remains independent and private.
Using IIFE solves multiple problems by providing scope isolation and immediate execution!
Very Important Questions!
Q1: How are variables and functions private in different modules?
A: Because of IIFE! The require statement wraps code inside IIFE, making everything private by default!
Module Privacy through IIFE:
============================
// What you write in sum.js:
const secret = "hidden";
function add(a, b) { return a + b; }
module.exports = add;
// What Node.js actually does:
(function(exports, require, module, __filename, __dirname) {
const secret = "hidden"; // Private! Can't access outside
function add(a, b) { return a + b; }
module.exports = add; // Only this is exported!
})();
Q2: How do you get access to module.exports? Where does this module come from?
A: In Node.js, when your code is wrapped inside a function, this function has a parameter named module. This parameter is an object provided by Node.js that includes module.exports.
// The wrapper function with parameters:
(function(exports, require, module, __filename, __dirname) {
// Your code here
// 'module' is passed as a parameter!
// 'module.exports' is how you export things
})();
When you use module.exports, you're modifying the exports object of the current module. Node.js relies on this object to determine what will be exported from the module when it's required in another file.
The module object is automatically provided by Node.js and is passed as a parameter to the function that wraps your code. This mechanism allows you to define which parts of your module are accessible externally.
How require() Works Behind the Scenes
This is very important to understand!
The require() Process:
======================
require('./xyz')
│
▼
┌─────────────────────┐
│ 1. RESOLVING │ ← Find the module path
└─────────────────────┘
│
▼
┌─────────────────────┐
│ 2. LOADING │ ← Load the file content
└─────────────────────┘
│
▼
┌─────────────────────┐
│ 3. WRAPPING (IIFE) │ ← Wrap in IIFE for privacy
└─────────────────────┘
│
▼
┌─────────────────────┐
│ 4. EVALUATION │ ← Run code, set module.exports
└─────────────────────┘
│
▼
┌─────────────────────┐
│ 5. CACHING │ ← Cache for future use
└─────────────────────┘
1. Resolving the Module
Node.js first determines the path of the module. It checks whether the path is a local file (./local), a JSON file (.json), or a module from the node_modules directory, among other possibilities.
2. Loading the Module
Once the path is resolved, Node.js loads the file content based on its type. The loading process varies depending on whether the file is JavaScript, JSON, or another type.
3. Wrapping Inside an IIFE
The module code is wrapped in an Immediately Invoked Function Expression (IIFE). This wrapping helps encapsulate the module's scope, keeping variables and functions private to the module.
4. Code Evaluation and Module Exports
After wrapping, Node.js evaluates the module's code. During this evaluation, module.exports is set to export the module's functionality or data. This step essentially makes the module's exports available to other files.
5. Caching (Very Important!)
Importance: Caching is crucial for performance. Node.js caches the result of the require() call so that the module is only loaded and executed once!
Caching Example - Very Important!
Scenario: Suppose you have three files: sum.js, app.js, and multiply.js. All three files require a common module named xyz.
project/
├── sum.js → require('./xyz')
├── app.js → require('./xyz')
├── multiply.js → require('./xyz')
└── xyz.js → The common module
Initial Require:
When sum.js first requires xyz with require('./xyz'), Node.js performs the full require() process for xyz:
- Resolving the path to xyz
- Loading the content of xyz
- Wrapping the code in an IIFE
- Evaluating the code and setting module.exports
- Caching the result!
Node.js creates a cached entry for xyz that includes the evaluated module exports.
Subsequent Requires:
When app.js and multiply.js later require xyz using require('./xyz'), Node.js skips the initial loading and evaluation steps. Instead, it retrieves the module from the cache!
Caching Flow:
=============
sum.js: require('./xyz')
→ Full process (resolve → load → wrap → evaluate → cache)
→ Returns module.exports
app.js: require('./xyz')
→ Check cache... FOUND!
→ Returns cached module.exports (FAST!)
multiply.js: require('./xyz')
→ Check cache... FOUND!
→ Returns cached module.exports (FAST!)
This means that for app.js and multiply.js, Node.js just returns the cached module.exports without going through the resolution, loading, and wrapping steps again!
Impact on Performance:
If caching did not exist, each require('./xyz') call would repeat the full module loading and evaluation process. This would result in a performance overhead, especially if xyz is a large or complex module and is required by many files.
With caching, Node.js efficiently reuses the module's loaded and evaluated code, significantly speeding up module resolution and reducing overhead!
Exploring the NodeJS GitHub Repository
Welcome back! Now, I will go to the Node.js GitHub repo to show you what's happening!
https://github.com/nodejs
1. Node.js is an Open-Source Project!
Yes, Node.js is completely open source. You can see all the code on GitHub!
2. V8 Engine Integration
I will now show you how the V8 JavaScript engine is integrated within the Node.js GitHub repository to illustrate its role and interaction with Node.js.
Node.js Repository Structure: ============================= nodejs/node/ ├── deps/ │ ├── v8/ ← V8 JavaScript Engine! │ ├── libuv/ ← Async I/O Library! │ └── ... ├── lib/ ← Built-in JS modules ├── src/ ← C++ source code └── ...
3. What are the Superpowers?
When I say there are superpowers, what are these superpowers? This is all the code for the superpowers in the deps folder!
4. libuv - The Most Amazing Superpower!
Node.js is popular just because of libuv!
libuv plays a critical role in enabling Node.js's high performance and scalability. It provides the underlying infrastructure for:
- Asynchronous I/O
- Event handling
- Cross-platform compatibility
The lib Directory - Built-in Modules
In the Node.js repository, if you navigate to the lib directory, you'll find the core JavaScript code for Node.js!
This lib folder contains the source code for various built-in modules like http, fs, path, and more. Each module is implemented as a JavaScript file within this directory.
nodejs/node/lib/ ├── fs.js ← File System module ├── http.js ← HTTP module ├── path.js ← Path module ├── os.js ← OS module ├── timers/ ← Timer functions │ └── promises.js ├── internal/ │ └── modules/ │ ├── helpers.js ← require implementation! │ └── cjs/ │ └── loader.js └── ...
Q: Where is setTimeout coming from and how does it work behind the scenes?
You can find it here:
https://github.com/nodejs/node/blob/main/lib/timers/promises.js
The timer functions like setTimeout, setInterval are defined in the timers module!
require() in NodeJS Repository
In helpers.js file, you can find the actual implementation of require method!
Here is where the require function is formed:
Path: node/lib/internal/modules/helpers.js
https://github.com/nodejs/node/blob/main/lib/internal/modules/helpers.js
makeRequireFunction
The makeRequireFunction creates a custom require function for a given module mod. This function:
- Validates that mod is an instance of Module
- Defines a require function that uses mod.require() to load modules
- Implements a resolve method for resolving module paths using Module._resolveFilename()
- Implements a paths method for finding module lookup paths using Module._resolveLookupPaths()
- Sets additional properties on the require function, such as main, extensions, and cache
// Simplified view of makeRequireFunction
function makeRequireFunction(mod) {
// Validate mod is a Module instance
function require(path) {
return mod.require(path);
}
require.resolve = function(request) {
return Module._resolveFilename(request, mod);
};
require.paths = function(request) {
return Module._resolveLookupPaths(request, mod);
};
require.main = process.mainModule;
require.extensions = Module._extensions;
require.cache = Module._cache;
return require;
}
Module Loader
https://github.com/nodejs/node/blob/main/lib/internal/modules/cjs/loader.js
If the id argument provided to the require() function is empty or undefined, Node.js will throw an exception!
This is because the require() function expects a string representing the path or identifier of the module to load. When it receives undefined instead, it results in a TypeError, indicating that an invalid argument value was provided.
require(); // TypeError!
require(undefined); // TypeError!
require(''); // Error!
// Correct usage:
require('./module'); // ✅
Node.js documentation and GitHub repository provide insights into how require() handles module loading. Reviewing these resources can help you understand how to properly use require() and handle potential errors.
Quick Recap
| Concept | Description |
|---|---|
| IIFE | Immediately Invoked Function Expression - wraps module code |
| Module Privacy | Variables are private because of IIFE wrapping |
| require() Steps | Resolve → Load → Wrap → Evaluate → Cache |
| Caching | Modules loaded once, cached for subsequent requires |
| libuv | The superpower behind Node.js async I/O |
| lib folder | Contains all built-in module source code |
Interview Questions
Q: How are variables and functions private in different modules?
"Node.js wraps each module's code inside an IIFE (Immediately Invoked Function Expression). This creates a function scope that keeps all variables and functions private unless explicitly exported using module.exports."
Q: Where does module.exports come from?
"When Node.js wraps module code in an IIFE, it passes 'module' as a parameter to that function. The module object is provided by Node.js and contains the exports property that we use to export things."
Q: What are the steps of require() behind the scenes?
"require() goes through 5 steps: 1) Resolving the module path, 2) Loading the file content, 3) Wrapping in IIFE, 4) Evaluating the code and setting module.exports, 5) Caching the result for future requires."
Q: Why is module caching important?
"Caching ensures that modules are only loaded and executed once. Subsequent require() calls return the cached module.exports, significantly improving performance and preventing redundant code execution."
Q: What is libuv and why is it important?
"libuv is a C library that provides Node.js with asynchronous I/O, event handling, and cross-platform compatibility. Node.js is popular largely because of libuv's ability to handle thousands of concurrent connections efficiently."
Key Points to Remember
- Module code is wrapped in IIFE
- IIFE provides scope isolation and immediate execution
- Variables are private by default because of IIFE
- module is passed as parameter to IIFE
- require() steps: Resolve → Load → Wrap → Evaluate → Cache
- Caching is crucial for performance
- Node.js is open source on GitHub
- libuv is the superpower behind async I/O
- lib folder contains built-in modules source code
- helpers.js contains makeRequireFunction
What's Next?
Now you understand how modules work behind the scenes in Node.js! In the next episode, we will:
- Learn about the File System (fs) module
- Read and write files
- Work with directories
Keep coding, keep learning! See you in the next one!
Post a Comment