Skip to content

Latest commit

 

History

History
85 lines (43 loc) · 4.05 KB

HISTORY.md

File metadata and controls

85 lines (43 loc) · 4.05 KB

V2.0.0 (May 2020)

  • Input of Personal Access Token (PAT) in Jupyter is not necessary any more

    While starting the kernel, the kernel manager will use an own kernel client and create the Spark Session and other artifacts via REST API and the secure SSH tunnels (DEMO).

  • Native Windows support

    Anaconda and Jupyter on Windows 10 (with OpenSSH) can be used with JupyterLab Integration (DEMO).

  • Docker support

    No need for local Anaconda and JupyterLab Integration installation - the quickest way to test JupyterLab Integration (DEMO).

  • Browsers

    • DBFS browser with file preview

      The DBFS browser does not use sidecar any more and allows to preview many text files like csv, sh, py, ... (DEMO)

    • Database browser with schema and data preview

      The Database browser does not use sidecar any more and allows to preview the table schema and shows sample rows of the data (DEMO)

    • MLflow browser

      A mlflow experiements browser that converts all runs of an experiment into a Pandas Dataframe to query and compare best runs in pandas. (DEMO - Intro), (DEMO - Keras), (DEMO - MLlib)

  • dbutils

    • Support for dbutils.secrets

      dbutils.secrets allow to hide credentials from your code (DEMO)

    • Support for dbutils.notebook

      Higher compatibility with Databricks notebooks:

      • dbutils.notebook.exit stops "Running all cells" (DEMO)
      • dbutils.notebook.run allows to run .pyand .ipynb files from notebooks in JupyterLab Integration (DEMO)

  • Support for kernels without Spark

    Create a JupyterLab Integration kernel specification with --nospark if no Spark Session on the remote cluster is required, e.g. for Deep Learning (DEMO)

  • Support of Databricks Runtimes 6.4 and higher (incl 7.0)

    The changed initialisation from DBR 6.4 and above (pinned mode) is now supported

  • JupyterLab 2.1 is now default

    Bumped JupyterLab to the latest version

  • Experimental features

    • Scala support (experimental)

      The %%scala magic will send Scala code to the same Spark Context (DEMO)

    • %fs support (experimental)

      The %fs of %%fs magic is supported as a shortcut for dbutils.fs.xxx (DEMO)

V1.0.x (December 2019)

  • Use Databricks CLI profiles and contain URLs and tokens

    Jupyterlab Integration used officially supported Databroicks CLI configurations to retrieve the Personal Access Tokens and URLs for remote cluster access. Personal Access Tokens will not be copied to the remote cluster

  • Create and manage Jupyter kernel specifications for remote Databricks clusters

    Jupyterlab Integration allows to create Jupyter kernel specifications for remote Databricks clusters via SSH. Kernel specifications can also be reconfigured or deleted

  • Configure SSH locally and remotely

    Jupyterlab Integration allows to create a local ssh key pair and configure the cluster with the public key for SSH access. INjecting the public key will restart the remote cluster

  • Create a Spark session and attach notebooks to it

    With Jupyterlab Integration, one needs to provide the Personal Access Token in th browser to authenticate the createion of a Spark Session. The current notebook will then be connected with this Spark session.

  • Mirror a a remote Databricks environment

    Jupyterlab Integration can mirror the versions of Data Science related libraries to a local conda environment. A blacklist and a whitelist allow to control which libraries are actually mirrored