aviemzur commented on a change in pull request #8: URL: https://github.com/apache/incubator-liminal/pull/8#discussion_r525126164
########## File path: docs/source/getting_started_with_liminal.md ########## @@ -0,0 +1,153 @@ + + +# Getting Started with Apache Liminal + +Liminal is a MachineLearning orchestration platform, or in other words: Liminal enables data scientists and data engineers to define their entire ML system and flow using configuration. From feature engineering to production monitoring - and the framework takes care of the infra behind the scenes and seamlessly integrates with the infrastructure behind the scenes. + + + +## Quick Start + +So, you’ve heard about liminal and decided you want to give it a try. + +This guide will allow you to set up your first apache Liminal environment and allow you to create some simple ML pipelines. These will be very similar to the ones you are going to build for real production scenarios. This is actually the magic behind Liminal. + + + +## Prerequisites + +Python 3 (3.6 and up) + +[Docker Desktop](https://www.docker.com/products/docker-desktop) + +*Note: make sure your kubernetes cluster is running* + +### Apache Liminal Hello World + +In this tutorial, we will go through setting up Liminal for the first time on your local dev machine. + +I’m running this on my macBook, so this will cover mac related installation aspects and kinks (if there are any). + +First, let’s build our examples project: + +In the dev folder, just clone the example code from liminal: + + +``` +git clone https://github.com/apache/incubator-liminal +``` +***Note:*** *You just cloned the entire Liminal Project, you actually need just the examples folder.* + +Create a python virtualenv to isolate your runs, like this: + + +``` +cd incubator-liminal/examples/liminal-getting-started +python3 -m venv env +``` + + +And activate your virtual environment: + + +``` +source ../env/bin/activate +``` + +Now we are ready to install liminal: + + +``` +pip uninstall apache-liminal +pip install apache-liminal +rm -rf ~/liminal_home +``` +Let's build the images you need for the example: +``` +liminal build +``` +The build will create docker images based on the Liminal.yml file, eash task and service will have its docker image created. + + +``` +liminal deploy --clean # the --clean is for removing old containers from the previous install +``` +The deploy will create missing containers from the images created above as well as basic Airflow containers needed to run the piplines, and deploy them to your local kubernetes cluster + +*Note: Liminal creates a bunch of assets, some are going to be located under: the directory in your LIMINAL_HOME env variable. But default LIMINAL_HOME is set to ~/liminal_home directory.* + +Now lets runs it: +``` +liminal start +``` +Liminal now spins up the AirFlow containers it will run the dag created based on the Liminal.yml file. Review comment: `AirFlow` -> `Apache Airflow` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
