Initial commit

README.md

@@ -0,0 +1,70 @@
+## About
+
+This is a minimal JavaScript reader for the [TRX tractography file format](https://github.com/tee-ar-ex/trx-spec/blob/master/specifications.md). Previously, most tractography tools used their own proprietary [format](https://www.nitrc.org/plugins/mwiki/index.php/surfice:MainPage#Supported_Formats) such as BFLOAT, niml.tract, PDB, [TCK](https://mrtrix.readthedocs.io/en/latest/getting_started/image_data.html#tracks-file-format-tck), [TRK](http://trackvis.org/docs/?subsect=fileformat) and VTK. The TRX format was developed from community [discussions](https://github.com/nipy/nibabel/issues/942) to address the [limitations of the existing formats](https://docs.google.com/document/d/1GOOlG42rB7dlJizu2RfaF5XNj_pIaVl_6rtBSUhsgbE/edit#heading=h.c6igqydj1hrf) and the needs of the users. This reader supports all the features of TRX including data-per-vertex, data-per-streamline and groups.
+
+## Live Demo
+
+[NiiVue provides a WebGL live demo of this code](https://niivue.github.io/niivue/features/tracts.html). This can be tested on any device (computer, laptop, phone). A sample TRX file is loaded by default, but users can drag and drop new streamlines in the niml.tract, TCX, TRK, TRX, and VTK formats.
+
+## Node.JS Command Line Demo
+
+The module `streamlineIO.mjs` provides JavaScript reading for streamlines in the TCX, TRK, TRX, and VTK formats. The included file `bench.mjs` demonstrates these capabilities by loading a streamline 11 times and reporting details regarding file size, loading time, and streamline properties.
+
+To run the demo:
+
+```
+$ git clone https://github.com/tee-ar-ex/trx-javascript
+$ cd trx-javascript
+$ npm install gl-matrix fflate fzstd
+$ node bench.mjs dpsv.trx
+dpsv.trx	Size	626180	Time	302
+Vertices:95865
+ First vertex (x,y,z):-24.25,-22.09375,-26.90625
+Streamlines: 460
+ Vertices in first streamline: 208
+dpg (data_per_group) items: 0
+dps (data_per_streamline) items: 1
+  'DataSetID' items: 460
+dpv (data_per_vertex) items: 1
+  'z' items: 95865
+Header (header.json):
+{
+  DIMENSIONS: [ 314, 378, 272 ],
+  VOXEL_TO_RASMM: [
+    [ 0.5, -0, 0, -78.5 ],
+    [ -0, 0.5, 0, -112.5 ],
+    [ -0, -0, 0.5, -50 ],
+    [ 0, 0, 0, 1 ]
+  ],
+  NB_VERTICES: 95865,
+  NB_STREAMLINES: 460
+}
+Done
+```
+
+## Implementation Details
+
+There are several important considerations regarding supporting the TRX format with JavaScript. The provided minimal reader makes some tradeoffs that may not be appropriate for all use cases.
+
+ - The TRX [specification](https://github.com/tee-ar-ex/trx-spec/blob/master/specifications.md) allows streamline positions to use the float16 datatype, which is not native to JavaScript. This code converts these to float32.
+ - Be aware that the specification stores NB_STREAMLINES values in the offsets array, with each value pointing to the start of that streamline One must use the length of the positions array or the header to infer the end of the final streamline. This code will populate return an offset array with NB_STREAMLINES+1 values to solve the [fencepost problem](https://icarus.cs.weber.edu/~dab/cs1410/textbook/3.Control/fencepost.html) for the final streamline. This simplifies and accelerates display code, but one must be aware of this modification.
+ - The TRX specification requires little-endian order. The current code only supports little-endian systems. This should support all modern Android, iOS, macOS, Linux and Windows devices.
+ - This tool uses [fflate](https://github.com/101arrowz/fflate) to decompress GZip and Zip files. During development we found that this library is much faster than the pako and jszip alternatives.
+
+## Benchmark
+
+The included JavaScript `bench` provides a method to evaluate performance. This benchmark is likely specific to JavaScript and so caution should be excercised in evaluating relative performance. The script will report the time to load a TRK, TCK, VTK or TRX file 10 times (it loads the tracts 11 times, and ignores the first run).
+
+The graph below shows the time load the [left inferior fronto-occipital fasciculus (IFOF) ](https://brain.labsolver.org/hcp_trk_atlas.html) with 30856 streamlines and 11437105 vertices. The benchmark was run on a passively-cooled 15w M1-based MacBook Air. The ideal format would be both fast to load (to the left on the horizontal axis) and have a small file size (toward the bottom in the right axis). However, compression typically trades load time for file size. Here all data is loaded from a local solid state drive, whereas smaller files would benefit if data was loaded using a slow internet connection. The following file formats are illustrated (except where noted, both positions and indeices are stored with 32-bit precision):
+
+ - vtk: streamlines saved by an [undocumented](https://discourse.vtk.org/t/upcoming-changes-to-vtkcellarray/2066) extension to the [VTK legacy file format](https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf).
+ - tck: [MRtrix format](https://mrtrix.readthedocs.io/en/latest/getting_started/image_data.html#tracks-file-format-tck).
+ - trk: The popular [TrackVis format](http://trackvis.org/docs/?subsect=fileformat).
+ - trk.gz: Gzip compressed TrackVis.
+ - trk.zst: ZStandard compressed TrackVis. Note that [native zstd decompression](https://github.com/neurolabusc/zlib-bench-python) is very fast, but the JavaScript decompressor is relatively slow.
+ - trx: uncompressed TRX format.
+ - z.trx: zip-compressed TRX format.
+ - 16.trx: uncompressed TRX format using 16-bit floats for positions (which are not native to JavaScript).
+ - 16z.trx: zip-compressed TRX format using 16-bit floats for positions (which are not native to JavaScript).
+
+![M1 Performance](M1.png)

bench.mjs

@@ -0,0 +1,70 @@
+//Install dependencies
+// npm install gl-matrix fflate fzstd
+//Test on tractogram
+// node bench.mjs dpsv.trx
+
+import * as streamline from "./streamlineIO.mjs";
+import * as fs from "fs";
+
+async function main() {
+    let argv = process.argv.slice(2);
+    let argc = argv.length;
+    if (argc < 1) {
+        console.log("arguments required: 'node bench.mjs dpsv.trx'")
+        return
+    }
+    //check input filename
+    let fnm = argv[0];
+    if (!fs.existsSync(fnm)) {
+        console.log("Unable to find NIfTI: " + fnm);
+        return
+    }
+    let re = /(?:\.([^.]+))?$/;
+    let ext = re.exec(fnm)[1];
+    ext = ext.toUpperCase();
+    let obj = [];
+    let d = Date.now();
+    let nrepeats = 11; //11 iterations, ignore first
+    for (let i = 0; i < nrepeats; i++) {
+        if (i == 1) d = Date.now(); //ignore first run for interpretting/disk
+        if (ext === "FIB" || ext === "VTK" || ext === "TCK" || ext === "TRK" || ext === "GZ" || ext === "ZSTD" || ext === "ZST") {
+            const buf = fs.readFileSync(fnm);
+            if (ext === "TCK")
+                obj = streamline.readTCK(new Uint8Array(buf).buffer);
+            else if (ext === "FIB" || ext === "VTK")
+                obj = streamline.readVTK(new Uint8Array(buf).buffer);
+            else
+                obj = streamline.readTRK(new Uint8Array(buf).buffer);
+        } else {
+            obj = await streamline.readTRX(fnm, true);
+        }
+    }
+    let ms = Date.now() - d;
+    //find file size:
+    let dat = fs.readFileSync(fnm);
+    console.log(`${fnm}\tSize\t${dat.length}\tTime\t${ms}`);
+    console.log("Vertices:" + obj.pts.length / 3);
+    console.log(" First vertex (x,y,z):" + obj.pts[0] + ',' + obj.pts[1] + ',' + obj.pts[2]);
+    console.log("Streamlines: " + (obj.offsetPt0.length - 1)); //-1 due to fence post
+    console.log(" Vertices in first streamline: " + (obj.offsetPt0[1] - obj.offsetPt0[0]));
+    if (obj.hasOwnProperty("dpg")) {
+        console.log("dpg (data_per_group) items: " + obj.dpg.length);
+        for (let i = 0; i < obj.dpg.length; i++)
+            console.log("  '" + obj.dpg[i].id + "' items: " + obj.dpg[i].vals.length);
+    }
+    if (obj.hasOwnProperty("dps")) {
+        console.log("dps (data_per_streamline) items: " + obj.dps.length);
+        for (let i = 0; i < obj.dps.length; i++)
+            console.log("  '" + obj.dps[i].id + "' items: " + obj.dps[i].vals.length);
+    }
+    if (obj.hasOwnProperty("dpv")) {
+        console.log("dpv (data_per_vertex) items: " + obj.dpv.length);
+        for (let i = 0; i < obj.dpv.length; i++)
+            console.log("  '" + obj.dpv[i].id + "' items: " + obj.dpv[i].vals.length);
+    }
+    if (obj.hasOwnProperty("header")) {
+        console.log("Header (header.json):");
+        console.log(obj.header);
+    }
+}
+main().then(() => console.log('Done'))

graphM1.py

@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+
+import pandas as pd
+from matplotlib import pyplot as plt
+
+# Set the figure size
+plt.rcParams["figure.figsize"] = [7.00, 3.50]
+plt.rcParams["figure.autolayout"] = True
+
+# Create a dataframe
+df = pd.DataFrame(
+   dict(
+      time=[2354, 3337, 3418, 11509, 15376, 494, 8161, 14928, 21013],
+      bytes=[217428604, 137615686, 137369684, 59896111, 38597377, 137369222, 62858566, 68746592, 53953006],
+      points=['vtk', 'tck', 'trk', 'trk.gz', 'trk.zst', 'trx', 'z.trx', '16.trx', '16z.trx']
+   )
+)
+
+# Scatter plot
+ax = df.plot.scatter(title='MacBook Air M1', x='time', y='bytes', alpha=0.5)
+ax.set_ylim(ymin=0)
+# Annotate each data point
+for i, txt in enumerate(df.points):
+   ax.annotate(txt, (df.time.iat[i], df.bytes.iat[i]))
+
+#plt.show()
+plt.savefig('M1.png', dpi=300)