Firetrace v2.0.1

• Increased model size. With the improved performance of PyTorch, we were able to significantly increase the size of the model without impacting storage. Firetrace now has 6 hidden layers, with 100 neurons in each. This brings the total parameter count to 11201. The model has been trained for 6250 epochs.

  Here is a sample set of results from the updated model.
  

  And a video of its performance during training
  

  output.mp4

• Used Leaky ReLU activation. We suspected with the release of v2.0.0 that the model may have been suffering from the "dying ReLU" problem. As a precautionary measure we decided to move to leaky ReLU and saw marginal improvements.

• Used weight decay with the AdamW optimiser. This was another precaution, this time to avoid overfitting the model as we scaled it up.

• The interface now shows orange fire colours, rather than purple.

---

• The HuggingFace repository is now divergent from the main GitHub one. A commit is manually created with every release.

Firetrace v2.0.0 - 2nd Generation, 1st Release

Confusingly, the previous release is labelled as model-v2, "the second generation of large Firetrace models". We have since revised our naming standards.

• model-v1 is now v1.0.0, the first model of the first generation of Firetrace models.
• model-v2 is now v1.1.0, the second model of the first generation of Firetrace models, featuring reduced model size over previous generations, in addition to the use of SOI data.

Introducing Firetrace v2.0.0

The following is a list of changes we made with v2.0.0, using the 2nd generation Firetrace model. It’s a long one 😅

Used PyTorch

We looked at some new libraries and technologies that had reached general availability during the time between the release of Firetrace v1.0.0 and when development started for v2.0.0.

We also noticed a few libraries that we overlooked during development of Firetrace v1.0.0

New libraries may come with performance improvements/additional features, so we thought that this part of the process was super important.

mlx - recently developed ML framework from Apple to run on Apple Silicon processors, making use of the Unified Memory architecture and Neural Engine. We ultimately decided against using it, as the HuggingFace servers do not use Apple hardware, and using mlx would severely limit the amount of users that could use Firetrace.

Mojo - newly released Python-compatible language specifically designed for AI/ML work. Mojo is not avaliable for Windows without WSL (as seems to be the case for most new languages), but this was decidedly not an issue for us as we primarily support deploying to HuggingFace’s Ubuntu servers. Unfortunately, Mojo has no AI ecosystem and relies on developers to write functions from scratch. This far exceeded our capabilities.

JAX/Flax/Equinox - Machine learning library/family of libraries maintained by Google Deepmind (ooh prestige). Specifically enhances support for TPU. Unfortunately JAX is extremely new to the AI world, and is almost exclusively used within Google research labs. This means that there is a lack of public documentation and examples, and its novel approach to creating models meant that it was extremely difficult to develop for.

✨ We are very interested in attempting to write in JAX as the ecosystem matures.

PyTorch - PyTorch had been around for many years before Firetrace started development. However, only within the previous 1 or 2 years has its popularity taken off. Compared to Tensorflow, it has better performance, and better interoperability/transferability between different hardware types (TPU, CUDA, MPS, AVX-512, and your usual CPU). It also offers the option to compile a limited set of functions to native code, boosting performance on training and inference.

Last minute change: We do not utilise torch.compile() on inference, as it doesn’t seem to work on HuggingFace servers. HuggingFace is the only platform Firetrace actively develops for currently. If you would like to change this, please open a pull request.

Updated how fire data was interpreted

Improving upon the previous model, with access to documentation for NASA's FIRMS product (https://www.earthdata.nasa.gov/learn/find-data/near-real-time/firms/mcd14dl-nrt#ed-firms-attributes), it should be noted that scan and track are the actual values for the width and length of each fire pixel.

To calculate the area of fire, we take $scan\times track$

We use data from NASA's MODIS satellite, for its accuracy. While VIIRS has a higher resolution, we do not need such precise measurements for the purposes of Firetrace, and we aren't ready to deal with the reduced accuracy (or rather - reduced signal-noise ratio) from VIIRS.

Improvements to pre-processing

Used faster pandas operations and reduced excessive/unnecessary operations on dataframes. This results in a 12x speedup to the pre-processing stage, allowing us to use larger datasets.

Improvements to the training process

Created a robust testing harness that can:

• Visualises the performance of the model over time
• Saves the best models, making it resistant to overfitting, and system crashes

Tensorflow had this functionality, but since migrating to PyTorch, we have had to rewrite most of these functions ourselves - allowing us to specifically build for the tasks that Firetrace requires, reducing bloat.

Visualisation Map 🔥🗺️

Given the limited data that we could possibly ask a user for, it is impossible for us to build a model that actually predicts the locations of specific fires.

Instead, we split Australia into a grid, and sample the amount of times fire appears in each cell for each given month, across our entire dataset.

This is what December looks like.

However, this set of data is too large to effectively deploy without impacting inference latency. We therefore take a random sample of 10,000 records in this dataset in order to make it lightweight enough for deployment.

After receiving a result from the model, we make another random sample of this dataset, before sorting each cell by frequency. We display the top x cells by frequency, based on the result given by the model. This allows for map results to display natural variations in fire patterns, while still remaining relatively realistic given the historical likelihood of fires in a given area.

model-v2

2nd generation of large models. includes SOI as a factor.

model-v1

introducing version 1 - an experimental model with short training times and a very large model size (16, 1024, 1024, 1).