source_detection_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "749b0ddf",
   "metadata": {
    "tags": []
   },
   "source": [
    "<img align=\"left\" src = https://project.lsst.org/sites/default/files/Rubin-O-Logo_0.png width=170 style=\"padding: 10px\"> \n",
    "<br><b>Little Demo: Source Detection</b> <br>\n",
    "Contact authors: Douglas Tucker, Melissa Graham<br>\n",
    "Last verified to run: 2023-09-22 <br>\n",
    "LSST Science Pipelines version: Weekly 2023_37 <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9da1a210-d858-42fe-8591-570965b8be1a",
   "metadata": {},
   "source": [
    "Rerun source detection on a processed visit image (`calexp`) and display the results.\n",
    "\n",
    "Import packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc506441-c5c1-4101-9090-3f2c59a04a37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import lsst.daf.base as dafBase\n",
    "from lsst.daf.butler import Butler\n",
    "import lsst.afw.display as afwDisplay\n",
    "import lsst.afw.table as afwTable\n",
    "from lsst.pipe.tasks.characterizeImage import CharacterizeImageTask\n",
    "from lsst.meas.algorithms.detection import SourceDetectionTask\n",
    "from lsst.meas.deblender import SourceDeblendTask\n",
    "from lsst.meas.base import SingleFrameMeasurementTask"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e6e035b-e492-42ed-b57a-ddbcff9a3220",
   "metadata": {},
   "source": [
    "Set plotting parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36096062-7851-4099-bd5a-ad35efacb434",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "plt.style.use('tableau-colorblind10')\n",
    "afwDisplay.setDefaultBackend('matplotlib')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ceb1ff23-a9c7-41cd-bc97-dd7e6c9be007",
   "metadata": {},
   "source": [
    "Instantiate an instance of the butler."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cc5b27f-c583-4480-8f18-6e4937cd289c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "butler = Butler(config=\"dp02\", collections=\"2.2i/runs/DP0.2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ba4499-ec3f-403c-9bfc-7f95b9c0d147",
   "metadata": {},
   "source": [
    "Obtain the `calexp` image for `detector` 75 in `visit` 512055.\n",
    "\n",
    "Also obtain the sources (`src`) detected in the image with SNR $> 5$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3ee02e9-9951-44af-a3d9-8a32bf61b8fd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dataId = {'visit': 512055, 'detector': 75}\n",
    "calexp = butler.get('calexp', **dataId)\n",
    "srcs = butler.get('src', **dataId)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fb661a3-8480-45c9-a5e2-f43f04151a32",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(len(srcs))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8baf781b-3486-4998-93da-59e30aa862c1",
   "metadata": {},
   "source": [
    "Display the `calexp`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80bb16af-003b-43d1-bafb-38dedb54f90e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(6, 6))\n",
    "afw_display = afwDisplay.Display(fig)\n",
    "afw_display.scale('asinh', 'zscale')\n",
    "afw_display.mtv(calexp.image)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f588d7dc-3b39-4561-adc8-0668721d51fa",
   "metadata": {},
   "source": [
    "Display the `calexp` and mark the $>2000$ detected sources with SNR $>5$ with orange circles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdedae7a-aa07-4418-a419-e904658d8e80",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(6, 6))\n",
    "afw_display = afwDisplay.Display(fig)\n",
    "afw_display.scale('asinh', 'zscale')\n",
    "afw_display.mtv(calexp.image)\n",
    "\n",
    "with afw_display.Buffering():\n",
    "    for s in srcs:\n",
    "        afw_display.dot('o', s.getX(), s.getY(), size=20, ctype='orange')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8634d4c4-f318-434f-b77c-e8ddef26f156",
   "metadata": {},
   "source": [
    "Before redoing source detection and measurement, clear the existing `DETECTED` mask plane."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "344af33c-1c87-4a2c-9630-34522bce34f1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "calexp.mask.removeAndClearMaskPlane('DETECTED')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caa35bd1-6e37-4aa9-b78d-87f958dee916",
   "metadata": {},
   "source": [
    "**Step 1: Configure the tasks.**\n",
    "\n",
    "Create a minimal schema for the source table, and a container to record metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9aa6787b-aaa5-44a3-ba6c-db3ad87a7bf4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "schema = afwTable.SourceTable.makeMinimalSchema()\n",
    "algMetadata = dafBase.PropertyList()\n",
    "algMetadata"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cf0992b-f964-40ff-8f65-1f1b1a6de733",
   "metadata": {},
   "source": [
    "Example of how to view the help for a task, or a task's configuration class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8603996-260e-4dd9-88f5-b1c74d1ff3e9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# help(charImageTask)\n",
    "# help(CharacterizeImageTask.ConfigClass())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddac4a8e-78d3-48c2-a6ea-3c05239c92b5",
   "metadata": {},
   "source": [
    "Configure the task to characterize the image properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a6de09c-1f84-47f7-8f55-626bf25c58fa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "config = CharacterizeImageTask.ConfigClass()\n",
    "config.psfIterations = 1\n",
    "charImageTask = CharacterizeImageTask(config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b25ea206-3d64-4480-abc7-b2a82c32e06c",
   "metadata": {},
   "source": [
    "Configure the task to detect sources in the image with SNR $> 10$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c06db9d4-c9b5-4b2e-b988-8348b2917f39",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "config = SourceDetectionTask.ConfigClass()\n",
    "config.thresholdValue = 10\n",
    "config.thresholdType = \"stdev\"\n",
    "sourceDetectionTask = SourceDetectionTask(schema=schema, config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45a840ce-86f0-4bfa-96a6-136c88e08037",
   "metadata": {},
   "source": [
    "Configure the task to deblend the detected sources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "097cb4fa-cd53-4a0e-aa88-0aee45daa205",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sourceDeblendTask = SourceDeblendTask(schema=schema)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff42a2f6-f835-4b89-8ffb-5e0a09c89d9c",
   "metadata": {},
   "source": [
    "Configure the task to measure the detected sources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c434d66c-4ddc-42af-8b69-ee25a67a04ed",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "config = SingleFrameMeasurementTask.ConfigClass()\n",
    "sourceMeasurementTask = SingleFrameMeasurementTask(schema=schema,\n",
    "                                                   config=config,\n",
    "                                                   algMetadata=algMetadata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "687db243-8ab9-428e-a7ac-fbbbd4c218d1",
   "metadata": {},
   "source": [
    "Create `SourceTable` to hold the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc8f87bf-eb85-46df-a917-2bf735b42f97",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tab = afwTable.SourceTable.make(schema)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d349fb9-c61b-465e-b381-bc2798c76acf",
   "metadata": {},
   "source": [
    "**Step 2: Run the tasks.**\n",
    "\n",
    "Characterize the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "375f681d-804f-494e-b073-f66c6d7f4c16",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "temp = charImageTask.run(calexp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d64ca72-ace2-4d30-a0fd-d0fdef5c93be",
   "metadata": {},
   "source": [
    "Run source detection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6566e107-559d-4eae-830d-e001e4fe42c1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "result = sourceDetectionTask.run(tab, calexp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "386d20f9-8234-4e0a-8bdf-c465fccb0b07",
   "metadata": {},
   "source": [
    "Option to examine `result`.\n",
    "\n",
    "A `Struct` is a generalized container for storing multiple output components and accessing them as attributes. \n",
    "Explore the content of this `Struct` with the `getDict` method.\n",
    "Print the number of sources detected (about 940)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1bfc6fae-3ebb-4b96-8ca0-7f187d5c0807",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# print(type(result))\n",
    "# print(' ')\n",
    "# for k, v in result.getDict().items():\n",
    "#     print(k, type(v))\n",
    "# print(' ')\n",
    "# print('Number of positive peaks: ', result.numPosPeaks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "667702b9-1f33-4ce1-aed5-2c278cfaeb0d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-09-23T23:32:18.621193Z",
     "iopub.status.busy": "2023-09-23T23:32:18.620145Z",
     "iopub.status.idle": "2023-09-23T23:32:18.626100Z",
     "shell.execute_reply": "2023-09-23T23:32:18.625243Z",
     "shell.execute_reply.started": "2023-09-23T23:32:18.621160Z"
    },
    "tags": []
   },
   "source": [
    "Extract the sources detected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "444d2882-7758-4389-9d63-cda648e1920d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sources = result.sources"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52c4d28d-a76a-4438-a07d-d3d3614828ed",
   "metadata": {},
   "source": [
    "Most of the columns for `sources` will be NaN until deblending and measurement are run.\n",
    "Display the table to see the NaNs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c278486f-c8f0-45f4-9c2f-1b7ce54975d4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# sources"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc659715-c636-43d8-b727-54d09936276f",
   "metadata": {},
   "source": [
    "Run deblending."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4587f17-fef0-409d-8e68-e18a079ee94e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sourceDeblendTask.run(calexp, sources)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f585d79c-a495-47af-b155-bd46f2a0d373",
   "metadata": {},
   "source": [
    "Run measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86e283b1-4b07-40c0-8e99-ef082ccc0e8c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sourceMeasurementTask.run(measCat=sources, exposure=calexp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86bd428c-f9e3-4e8b-b300-e65a31f1d8ab",
   "metadata": {},
   "source": [
    "**Step 3: Visualize the detected sources.**\n",
    "\n",
    "Display the `calexp` and mark the $\\sim940$ detected sources with SNR $> 10$ with cyan circles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a40ba5d-4692-4396-95a2-51bdc14f532e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(6, 6))\n",
    "afw_display = afwDisplay.Display(fig)\n",
    "afw_display.scale('asinh', 'zscale')\n",
    "afw_display.mtv(calexp.image)\n",
    "\n",
    "with afw_display.Buffering():\n",
    "    for s in sources:\n",
    "        afw_display.dot('o', s.getX(), s.getY(), size=20, ctype='cyan')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e071a046-bb3c-4bf3-a667-14f1bbbd3996",
   "metadata": {},
   "source": [
    "Option to save the sources to disk as a FITS table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a04e1fc-418c-4195-992f-7809bcfc5223",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# sources.writeFits(\"my_sources.fits\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25efd928-73e2-47e4-a2e2-099ab934634a",
   "metadata": {},
   "source": [
    "**Exercise:** Rerun source detection after setting `config.thresholdValue = 3` and find about 5340 positive peaks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "015059b0-b448-4c51-b56c-9f0f09800378",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}