demo/pub_projet_research_hal_no_oa.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3c07384",
   "metadata": {},
   "source": [
    "# Identifier dans HAL les publications issues des projets de recherche qui ne sont pas en accès ouvert"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c98b1ae",
   "metadata": {},
   "source": [
    "- extraire depuis HAL le nombre de publications d'une institution issues de projet ANR\n",
    "- extraire le nombre de ces publications qui ne sont pas en accès ouvert\n",
    "- calculer le pourcentage que cela représente\n",
    "- extraire la liste de ces publications avec les données : année, doctype, url, titre, submittedType\n",
    "- produire un tableau de ces publications afin de les traiter\n",
    "- exporter en .csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2fbb88e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests, json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f51671b0",
   "metadata": {},
   "source": [
    "## Nombre de publications dans HAL d'un étab. issues de projet ANR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8b1ac26d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2445"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# memo univ bordeaux 259761\n",
    "req = requests.get(f\"https://api.archives-ouvertes.fr/search/?q=structId_i:116256&fq=anrProject_t:['' TO *]&rows=0\")\n",
    "req_json = req.json()\n",
    "nb = req_json[\"response\"][\"numFound\"]\n",
    "nb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e7db227",
   "metadata": {},
   "source": [
    "## Nombre de publications projet ANR sans OA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "77cbce5a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "680"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "req = requests.get(f\"https://api.archives-ouvertes.fr/search/?q=structId_i:116256&fq=anrProject_t:['' TO *]&rows=0\\\n",
    "&fq=openAccess_bool:false\")\n",
    "req_json = req.json()\n",
    "nb_no_oa = req_json[\"response\"][\"numFound\"]\n",
    "nb_no_oa"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff3ee222",
   "metadata": {},
   "source": [
    "## Déduire le pourcentage de publication à ouvrir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "030457b6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27.8 % de publications issues de projet ANR ne sont pas en accès ouvert\n"
     ]
    }
   ],
   "source": [
    "print(f\"{round(nb_no_oa/nb*100,1)} % de publications issues de projet ANR ne sont pas en accès ouvert\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51e7eb7f",
   "metadata": {},
   "source": [
    "## Extraire la liste de ces publications à traiter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2255e1ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "req = requests.get(f\"https://api.archives-ouvertes.fr/search/?q=structId_i:116256&fq=anrProject_t:['' TO *]\\\n",
    "&fq=openAccess_bool:false&fl=publicationDateY_i,uri_s,title_s,submitType_s,docType_s&rows=10000\")\n",
    "req_json = req.json()\n",
    "pub_liste = req_json[\"response\"][\"docs\"]\n",
    "#pub_liste # pour afficher la liste des publications"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e963ce62",
   "metadata": {},
   "source": [
    "## Produire un tableau des publications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "09eb7544",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "329e1601",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title_s</th>\n",
       "      <th>uri_s</th>\n",
       "      <th>submitType_s</th>\n",
       "      <th>docType_s</th>\n",
       "      <th>publicationDateY_i</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[Mondes et représentations. Le point de vue du...</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-01753303</td>\n",
       "      <td>notice</td>\n",
       "      <td>COUV</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[Age differences in information finding tasks:...</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-01799994</td>\n",
       "      <td>notice</td>\n",
       "      <td>ART</td>\n",
       "      <td>2012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[Factorisation archétypale en matrices non-nég...</td>\n",
       "      <td>https://hal-amu.archives-ouvertes.fr/hal-01334618</td>\n",
       "      <td>notice</td>\n",
       "      <td>COMM</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             title_s  \\\n",
       "0  [Mondes et représentations. Le point de vue du...   \n",
       "1  [Age differences in information finding tasks:...   \n",
       "2  [Factorisation archétypale en matrices non-nég...   \n",
       "\n",
       "                                               uri_s submitType_s docType_s  \\\n",
       "0      https://hal.archives-ouvertes.fr/hal-01753303       notice      COUV   \n",
       "1      https://hal.archives-ouvertes.fr/hal-01799994       notice       ART   \n",
       "2  https://hal-amu.archives-ouvertes.fr/hal-01334618       notice      COMM   \n",
       "\n",
       "   publicationDateY_i  \n",
       "0                2016  \n",
       "1                2012  \n",
       "2                2016  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(pub_liste)\n",
    "df[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9c7c87fc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>publicationDateY_i</th>\n",
       "      <th>docType_s</th>\n",
       "      <th>uri_s</th>\n",
       "      <th>title_s</th>\n",
       "      <th>submitType_s</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2012</td>\n",
       "      <td>ART</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-01799994</td>\n",
       "      <td>[Age differences in information finding tasks:...</td>\n",
       "      <td>notice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>403</th>\n",
       "      <td>2021</td>\n",
       "      <td>ART</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-03317630</td>\n",
       "      <td>[Propositional proof systems based on maximum ...</td>\n",
       "      <td>notice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>386</th>\n",
       "      <td>2015</td>\n",
       "      <td>COMM</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-03007272</td>\n",
       "      <td>[Résultats préliminaires et perspectives des é...</td>\n",
       "      <td>notice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>323</th>\n",
       "      <td>2021</td>\n",
       "      <td>COUV</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-03440788</td>\n",
       "      <td>[Systèmes techniques et productions symbolique...</td>\n",
       "      <td>notice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>537</th>\n",
       "      <td>2020</td>\n",
       "      <td>ART</td>\n",
       "      <td>https://hal.archives-ouvertes.fr/hal-03317623</td>\n",
       "      <td>[Optimum stable model search: algorithms and i...</td>\n",
       "      <td>notice</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     publicationDateY_i docType_s  \\\n",
       "1                  2012       ART   \n",
       "403                2021       ART   \n",
       "386                2015      COMM   \n",
       "323                2021      COUV   \n",
       "537                2020       ART   \n",
       "\n",
       "                                             uri_s  \\\n",
       "1    https://hal.archives-ouvertes.fr/hal-01799994   \n",
       "403  https://hal.archives-ouvertes.fr/hal-03317630   \n",
       "386  https://hal.archives-ouvertes.fr/hal-03007272   \n",
       "323  https://hal.archives-ouvertes.fr/hal-03440788   \n",
       "537  https://hal.archives-ouvertes.fr/hal-03317623   \n",
       "\n",
       "                                               title_s submitType_s  \n",
       "1    [Age differences in information finding tasks:...       notice  \n",
       "403  [Propositional proof systems based on maximum ...       notice  \n",
       "386  [Résultats préliminaires et perspectives des é...       notice  \n",
       "323  [Systèmes techniques et productions symbolique...       notice  \n",
       "537  [Optimum stable model search: algorithms and i...       notice  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ordered = df[[\"publicationDateY_i\", \"docType_s\", \"uri_s\", \"title_s\", \"submitType_s\" ]]\n",
    "df_ordered.sample(5)\n",
    "# exporter en CSV"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "750383bb",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "## Discussion\n",
    "\n",
    "Quelles sont les limites ? \n",
    "\n",
    "Comment améliorer l'identification ? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2312c1dd",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}