Page tree
Skip to end of metadata
Go to start of metadata

General Guidelines

  • Use virtual environment to manage dependencies / libraries
  • Use requirements.txt to store list of dependencies
  • "pip freeze > requirements.txt" to create requirements.txt file with needed libraries
  • "pip install -r requirements.txt" to install the libraries in your project environment
  • Version control should always be used to manage source code, including trained models, scripts, training data and project configurations. Aalto uses version.aalto.fi Git version control system
  • Jupyter Lab / Notebook is used for model development during experimentation phase. Final deployable model files should be converted to Python files (.py)

Install Python environment

Install Anaconda distribution which includes most of the libraries needed for data science work. "The open-source Anaconda Distribution is the easiest way to perform Python/R data science and machine learning on Linux, Windows, and Mac OS X. With over 15 million users worldwide, it is the industry standard for developing, testing, and training on a single machine, enabling individual data scientists to:"

https://www.anaconda.com/distribution/#download-section

Use pip tool to install additional libraries. "pip install package-name"

Multiple Python distributions

You can have multiple Python distributions installed on same computer.

You can use conda tool to install additional Python environments. Execute following command in the shell.

conda create -n py38 python=3.8

You can activate the specific Python distribution by following command:

conda activate py38

And deactivate by:

conda deactivate

You can check the Python version by:

python --version

Installing Git version control

Aalto ITS - Machine Learning projects are stored on version.aalto.fi and can be accessed after the required permission is granted by the project owner. A Git desktop client can be used to clone the remote repositories to the local machine and thereafter make push/pull requests. Please follow the following instructions to get started with Git and connect with version.aalto.fi

Setting up virtual environment

Using virtual environments is important as it helps to maintain your system clean since you don’t install system-wide libraries that you are only going to need in a small project. It allows you to use a certain version of a library for one project and another version for another project: if you install the library system-wide and don’t use venv, then you can only use one version of the library

To get started with virtual environments:

  1. Install virtual environment with pip

    1. py -m pip install --user virtualenv

  2. Execute "python -m venv myvirtualdirectory" to create a virtual environment under your project directory

    1. NOTE: Ensure you have the correct Python distribution activated when you create the virtual environment, as this will be used in the virtual environment configuration (pyvenv.cfg). You can activate the python environment with: 
      conda activate py38
  3. Activate virtual environment

    1. Go to your virtual directory: cd myvirtualdirectory

    2. (Optionally) Execute "Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass"

    3. Execute "Scripts\activate" to activate the virtual environment

  4. Install packages within your virtual environment

    1. pip install -r requirements.text (installs the libraries defined in your project requirements.txt file)

  5. Work and develop code in your virtual environment

  6. If you want to switch projects or otherwise leave your virtual environment, simply run:

    1. run: "Deactivate"

For more information see links below:

Using virtual environment with Python notebook

In order to use Jupyter Notebook/Lab with virtual environment (and the libraries / dependencies in the virtual environment) follow these steps: 

  1. Go to your virtual environment and activate environment

  2. Run "jupyter lab" in your virtual environment to start Jupyter Lab

See https://anbasile.github.io/programming/2017/06/25/jupyter-venv/ for more information

Check installations

  1. Check jupyter version "jupyter --version"
  2. Check python version "py --version"

Common problems

https://github.com/jupyter/notebook/issues/4909

  • No labels