**Assignment 3 : Volume Rendering and Neural Radiance Fields** **Author:** *Rishikesh Jadhav (119256534)* Differentiable Volume Rendering ================ - Goal: Implement ray sampling for generating world-space rays and points along those rays. - Implementation: Ray sampling has been successfully implemented, allowing the generation of world-space rays and point sampling. Ray sampling ------------------------   ### Implementation Note The ray sampling task has been successfully implemented, allowing the generation of world-space rays and sampling of points along those rays. Point sampling ------------------------  ### Implementation Note - Goal: Implement stratified sampling of points along ray directions within a specific depth range. - Implementation: Key components include implementing the StratifiedSampler class for distance generation and point sampling. Volume rendering ------------------------   ### Implementation Note - Goal: Enable volume rendering and depth mapping using an emission-absorption (EA) model. - Implementation: Volume rendering has been successfully implemented, rendering color and depth information from volumetric data. Optimizing a basic implicit volume ================ - Goal: Optimize volumetric parameters using the differentiable volume renderer. - Implementation: Random ray sampling enables parameter optimization. Random ray sampling ------------------------ - Goal: Improve the loss function in the training process for more effective optimization. - Implementation: Replaced the loss function with mean squared error (MSE). Loss and training ------------------------ - Goal: Improve the loss function in the training process for more effective optimization. - Implementation: Replaced the loss function with mean squared error (MSE). - After training, the Box center is approximately (0.2502, 0.2506, -0.0005). - The Box side lengths are approximately (2.0051, 1.5035, 1.5032). Visualization ------------------------   ### Implementation Note The final goal is visualization and aims to assess the quality of the optimization process. The primary objective is to compare and generate a visual representation of the optimized volume using a spiral sequence Optimizing a Neural Radiance Field ================ This task involves implementing an implicit volume in the form of a Multi-Layer Perceptron (MLP) within the context of a Neural Radiance Field (NeRF). The MLP maps 3D positions to volume density and color. Visualization ------------------------  ### Implementation Note - Implemented MLP for Volume Mapping: Implemented the NeuralRadianceField class in implicit.py. The forward method of this class takes a RayBundle as input, and the MLP within the class computes density and color values for each sample point. This allows the neural network to map 3D positions to volume density and color. - Loss Function Implementation: Implemented the loss function in the train_nerf method. The loss function calculates the difference between predicted and ground truth color values and is used for training the implicit volume. - Scene Optimization with NeRF: The provided data loading, training, and checkpointing mechanisms in the codebase are utilized to optimize the scene. The MLP within the NeuralRadianceField class is responsible for mapping 3D positions to density and color values, which are optimized based on the provided images. This Task enables the implementation and optimization of a Neural Radiance Field (NeRF) using an MLP-based implicit volume, guiding the project towards improved scene reconstruction and rendering References ================ [848F-Assignment 3 Repository](https://github.com/848f-3DVision/assignment3) : Used for the README file and starter code. [PyTorch Official Documentation](https://pytorch.org/) [Pytorch 3D documentation](https://pytorch3d.org/)