Skip to content

Commit

Permalink
[Docs] Refine language (mlrun#2496)
Browse files Browse the repository at this point in the history
  • Loading branch information
jillnogold authored Oct 23, 2022
1 parent 30c86b0 commit d3768f4
Show file tree
Hide file tree
Showing 10 changed files with 410 additions and 279 deletions.
8 changes: 4 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

(architecture)=
<a id="architecture"></a>
## MLRun Architecture
# MLRun architecture

MLRun started as a community effort to map the different components in the ML project lifecycle, provide a common metadata layer, and automate the operationalization process (a.k.a MLOps).

Expand All @@ -19,7 +19,7 @@ production pipelines with minimal engineering efforts.
<p align="center"><img src="_static/images/mlrun-flow.png" alt="mlrun-flow" width="600"/></p><br>


### MLRun Deployment
## MLRun deployment

MLRun has two main components, the service and the client (SDK):

Expand All @@ -29,7 +29,7 @@ MLRun has two main components, the service and the client (SDK):
<p align="center"><img src="_static/images/mlrun-cluster.png" alt="mlrun-flow" width="600"/></p><br>


### MLRun: An Integrated and Open Approach
## MLRun: an integrated and open approach

Data preparation, model development, model and application delivery, and end to end monitoring are tightly connected:
they cannot be managed in silos. This is where MLRun MLOps orchestration comes in. ML, data, and DevOps/MLOps teams
Expand Down
81 changes: 43 additions & 38 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
# Using MLRun

```{div} full-width
MLRun is an open MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications, significantly reducing engineering efforts, time to production, and computation resources.
MLRun is an open MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications. MLRun significantly reduces engineering efforts, time to production, and computation resources.
With MLRun, you can choose any IDE on your local machine or on the cloud. MLRun breaks the silos between data, ML, software, and DevOps/MLOps teams, enabling collaboration and fast continuous improvements.
Get started with MLRun [**Tutorials and Examples**](./tutorial/index.html), [**Installation and setup guide**](./install.html), or read about [**MLRun Architecture**](./architecture.html)
Get started with MLRun **{ref}`Tutorials and examples <tutorial>`**, **{ref}`Installation and setup guide <install-setup-guide>`**, or read about **{ref}`architecture`**.
This page explains how MLRun addressed the [**MLOps Tasks**](#mlops-tasks) and the [**MLRun core components**](#core-components)
This page explains how MLRun addresses the [**MLOps tasks**](#mlops-tasks) and presents the [**MLRun core components**](#core-components).
```


<a id="mlops-tasks"></a>
## MLOps Tasks
## MLOps tasks

`````{div} full-width
Expand Down Expand Up @@ -59,67 +59,72 @@ Project access can be restricted to a set of users and roles.
{bdg-link-primary-line}`more... <./projects/project.html>`
{octicon}`mortar-board` **Docs:**
{bdg-link-info}`Projects and Automation <./projects/project.html>`
{bdg-link-info}`CI/CD Integration <./projects/ci-integration.html>`
{bdg-link-info}`Projects and automation <./projects/project.html>`
{bdg-link-info}`CI/CD integration <./projects/ci-integration.html>`
, {octicon}`code-square` **Tutorials:**
{bdg-link-primary}`quick start <./tutorial/01-mlrun-basics.html>`
{bdg-link-primary}`Automated ML Pipeline <./tutorial/04-pipeline.html>`
{bdg-link-primary}`Automated ML pipeline <./tutorial/04-pipeline.html>`
, {octicon}`video` **Videos:**
{bdg-link-warning}`quick start <https://youtu.be/xI8KVGLlj7Q>`
{bdg-link-warning}`Quick start <https://youtu.be/xI8KVGLlj7Q>`
### Ingest and process data
MLRun provides abstract interfaces to various offline and online [**data sources**](./concepts/data-feature-store.html), supports batch or realtime data processing at scale, data lineage and versioning, structured and unstructured data, and more.
In addition, the MLRun [**Feature Store**](./feature-store/feature-store.html) automates the collection, transformation, storage, catalog, serving, and monitoring of data features across the ML lifecycle and enables feature reuse and sharing.
In addition, the MLRun [**Feature store**](./feature-store/feature-store.html) automates the collection, transformation, storage, catalog, serving, and monitoring of data features across the ML lifecycle and enables feature reuse and sharing.
{bdg-link-primary-line}`more... <./data-prep/index.html>`
{octicon}`mortar-board` **Docs:**
{bdg-link-info}`Feature Store <./feature-store/feature-store.html>`
{bdg-link-info}`Data & Artifacts <./concepts/data-feature-store.html>`
{bdg-link-info}`Feature store <./feature-store/feature-store.html>`
{bdg-link-info}`Data & artifacts <./concepts/data-feature-store.html>`
, {octicon}`code-square` **Tutorials:**
{bdg-link-primary}`quick start <./tutorial/01-mlrun-basics.html>`
{bdg-link-primary}`Feature Store <./feature-store/basic-demo.html>`
{bdg-link-primary}`Feature store <./feature-store/basic-demo.html>`
### Develop and train models
MLRun allows you to easily build ML pipelines that take data from various sources or the Feature Store and process it, train models at scale with multiple parameters, test models, tracks each experiments, register, version and deploy models, etc. MLRun provides scalable built-in or custom model training services, integrate with any framework and can work with 3rd party training/auto-ML services. You can also bring your own pre-trained model and use it in the pipeline.
MLRun allows you to easily build ML pipelines that take data from various sources or the Feature Store and process it, train models at scale with multiple parameters, test models, tracks each experiments, register, version and deploy models, etc. MLRun provides scalable built-in or custom model training services that integrate with any framework and can work with 3rd party training/auto-ML services. You can also bring your own pre-trained model and use it in the pipeline.
{bdg-link-primary-line}`more... <./development/index.html>`
{octicon}`mortar-board` **Docs:**
{bdg-link-info}`Model Training and Tracking <./development/model-training-tracking.html>`
{bdg-link-info}`Batch Runs and Workflows <./concepts/runs-workflows.html>`
{bdg-link-info}`Model training and tracking <./development/model-training-tracking.html>`
{bdg-link-info}`Batch runs and workflows <./concepts/runs-workflows.html>`
, {octicon}`code-square` **Tutorials:**
{bdg-link-primary}`Train & Eval Models <./tutorial/02-model-training.html>`
{bdg-link-primary}`Automated ML Pipeline <./tutorial/04-pipeline.html>`
{bdg-link-primary}`Train & eval models <./tutorial/02-model-training.html>`
{bdg-link-primary}`Automated ML pipeline <./tutorial/04-pipeline.html>`
, {octicon}`video` **Videos:**
{bdg-link-warning}`Train & compare models <https://youtu.be/bZgBsmLMdQo>`
### Deploy models and applications
MLRun rapidly deploys and manages production-grade real-time or batch application pipelines using elastic and resilient serverless functions. MLRun addresses the entire ML application: intercepting application/user requests, running data processing tasks, inferencing using one or more models, driving actions, and integrating with the application logic.
{bdg-link-primary-line}`more... <./deployment/index.html>`
{octicon}`mortar-board` **Docs:**
{bdg-link-info}`Realtime Pipelines <./serving/serving-graph.html>`
{bdg-link-info}`Batch Inference <./concepts/TBD.html>`
{bdg-link-info}`Realtime pipelines <./serving/serving-graph.html>`
{bdg-link-info}`Batch inference <./concepts/TBD.html>`
, {octicon}`code-square` **Tutorials:**
{bdg-link-primary}`Realtime Serving <./tutorial/03-model-serving.html>`
{bdg-link-primary}`Batch Inference <./tutorial/07-batch-infer.html>`
{bdg-link-primary}`Advanced Pipeline <./tutorial/07-batch-infer.html>`
{bdg-link-primary}`Realtime serving <./tutorial/03-model-serving.html>`
{bdg-link-primary}`Batch inference <./tutorial/07-batch-infer.html>`
{bdg-link-primary}`Advanced pipeline <./tutorial/07-batch-infer.html>`
, {octicon}`video` **Videos:**
{bdg-link-warning}`Serve pre-trained models <https://youtu.be/OUjOus4dZfw>`
### Monitor and alert
Observability is built into the different MLRun objects (data, functions, jobs, models, pipelines, etc.), eliminating the need for complex integrations and code instrumentation. With MLRun, you can observe the application/model resource usage and model behavior (drift, performance, etc.), define custom app metrics, and trigger alerts or retraining jobs.
{bdg-link-primary-line}`more... <./monitoring/index.html>`
{octicon}`mortar-board` **Docs:**
{bdg-link-info}`Model Monitoring Overview <./monitoring/model-monitoring-deployment.html>`
{bdg-link-info}`Model monitoring overview <./monitoring/model-monitoring-deployment.html>`
, {octicon}`code-square` **Tutorials:**
{bdg-link-primary}`Model Monitoring & Drift Detection <./tutorial/05-model-monitoring.html>`
{bdg-link-primary}`Model monitoring & drift detection <./tutorial/05-model-monitoring.html>`
`````

<a id="core-components"></a>
## MLRun Core components
## MLRun core components

MLRun includes the following major components

Expand All @@ -128,37 +133,37 @@ MLRun includes the following major components
````{grid} 6
:gutter: 2
```{grid-item-card} Project Management & Automation (SDK, API, etc.)
```{grid-item-card} Project management & automation (SDK, API, etc.)
:columns: 12
:text-align: center
:link: ./projects/project.html
```
```{grid-item-card} Serverless Functions
```{grid-item-card} Serverless functions
:columns: 6 4 4 2
:text-align: center
:link: ./runtimes/functions.html
```
```{grid-item-card} Data & Artifacts
```{grid-item-card} Data & artifacts
:columns: 6 4 4 2
:text-align: center
:link: ./concepts/data.html
```
```{grid-item-card} Feature Store
```{grid-item-card} Feature store
:columns: 6 4 4 2
:text-align: center
:link: ./feature-store/feature-store.html
```
```{grid-item-card} Batch Runs & Workflows
```{grid-item-card} Batch runs & workflows
:columns: 6 4 4 2
:text-align: center
:link: ./concepts/runs-workflows.html
```
```{grid-item-card} Real-time Pipelines
```{grid-item-card} Real-time pipelines
:columns: 6 4 4 2
:text-align: center
:link: ./serving/serving-graph.html
Expand All @@ -172,19 +177,19 @@ MLRun includes the following major components
````
[**Project Management:**](./projects/project.html) A service (API, SDK, DB, UI) that manages the different project assets (data, functions, jobs, workflows, secrets, etc.) and provides central control and metadata layer.
**{ref}`Project management <projects>`:** A service (API, SDK, DB, UI) that manages the different project assets (data, functions, jobs, workflows, secrets, etc.) and provides central control and metadata layer.
[**Serverless Functions:**](./runtimes/functions.html) automatically deployed software package with one or more methods and runtime-specific attributes (such as image, libraries, command, arguments, resources, etc.)
**{ref}`Serverless functions <Functions>`:** automatically deployed software package with one or more methods and runtime-specific attributes (such as image, libraries, command, arguments, resources, etc.)
[**Data & Artifacts:**](./concepts/data-feature-store.html) Glueless connectivity to various data sources, metadata management, catalog, and versioning for structures/unstructured artifacts.
**{ref}`Data & artifacts <data-feature-store>`:** Glueless connectivity to various data sources, metadata management, catalog, and versioning for structures/unstructured artifacts.
[**Feature Store:**](./feature-store/feature-store.html) automatically collects, prepares, catalogs, and serves production data features for development (offline) and real-time (online) deployment using minimal engineering effort.
**{ref}`Feature store <feature-store>`:** automatically collects, prepares, catalogs, and serves production data features for development (offline) and real-time (online) deployment using minimal engineering effort.
[**Batch Runs & Workflows:**](./concepts/runs-workflows.html) Execute one or more functions with specific parameters and collect, track, and compare all their results and artifacts.
**{ref}`Batch Runs & workflows <workflows>`:** Execute one or more functions with specific parameters and collect, track, and compare all their results and artifacts.
[**Real-Time Serving Pipeline:**](./serving/serving-graph.html) Rapid deployment of scalable data and ML pipelines using real-time serverless technology, including API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc.
**{ref}`Real-time serving pipeline <serving-graph>`:** Rapid deployment of scalable data and ML pipelines using real-time serverless technology, including API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc.
[**Real-Time monitoring:**](./monitoring/index.html) monitors data, models, resources, and production components and provides a feedback loop for exploring production data, identifying drift, alerting on anomalies or data quality issues, triggering retraining jobs, measuring business impact, etc.
**{ref}`Real-time monitoring <monitoring>`:** monitors data, models, resources, and production components and provides a feedback loop for exploring production data, identifying drift, alerting on anomalies or data quality issues, triggering retraining jobs, measuring business impact, etc.
`````

Expand Down
1 change: 1 addition & 0 deletions docs/serving/writing-custom-steps.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"(writing-custom-steps)=\n",
"# Writing custom steps\n",
"\n",
"<!-- Add overview -->"
Expand Down
Loading

0 comments on commit d3768f4

Please sign in to comment.